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Presented in this report are results from a study 
"conducted to evaluate the impact of Project Developmental Continuity 
on children's development and academic achievement . Following the 
first chapter's introductory, overview of'the sludy's objectives and 
• organization ,v chapter 2 documents the data collection methods and 
summarizes the analysis questions addressed in the ' remainder of the 
report. Chapter. 3 presents the results of analyses designed to 
highlight the nature of the sample and the characteristics of the 
various instruments, including a presentation of descriptive data 
taken on the sample and an analysis of attrition patterns and the 
effects of attrition on the characteristics of the sample. 
Information about the instruments is then presented in terms of the 
distributions of scores, internal consistency and stability of 
individual measures, the factor structu're of the battery, and the 
relationships of the measures to criteria of "social competence." 
Chapter 4 examines treatment effects, presenting findings of program 
impact separately fo;: English-dominant and Spanish-dominant samples. 
Finally, all results are summarized in chapter 5. A description of 
the instruments used in the study is appended* (MP) 
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INTRODUCTION 



Overview of the Evaluation 

:Project Developmental Continuity_ began in 1974 as 

a concerted effort to demonstrate tlie value ,of coordinating 
Head Start and elementary school programs. Fifteen Head Start ^ 
grantees were selected to establish administrative linkages 
between Head .Start centers and elemeVitary schools within their _ 
communities, and to design programs that would provide continuity 
of progralh and services. Twelve sites are still participating 
in the program and eleven are included in the evaluation. 

The evaluation of this national demonstration effort was 
planned in two phases. In the first phase (1974 to 197.8) 
a careful study was made to detem. ne whether adequate conditions 
existed for carrying out a longituuinal study of the program's' 
impact- This study would focus on a single cohort of children 
who began Head Start in fall 1976 and would complete third grade 
in the spring of 19 81. Findings from the feasibility study-- 
summarized in a final report (Love, Granville, & Smith, 1978) 
and detailed in Interim Report VII (Granville, Love, & Morri-s , 
1977) —indicated that the child measures: (a) were sufficiently 
reliable and valid, (b) yielded scores that were sensitive to 
change beyond that which is simply a function of maturation, 
(c) related to teachers' and testers' judgments of the children s 
"social competence '• (d) would be suitable for use with children 
in grades K-3, and >e) were generally suitable for administration 
by paraprofessional testers. The feasibility study also in- 
dicated that the evaluation sample was suitable in two main 
respects: (a) the PDC, and comparison group samples appeared 
highly similar in terms of children's background character- 
istics and entering test performance; and (b) there was evidence 
from school records that there would be sufficient retention 
of the sample children over a five-year period so that longi- 
tudinal impact analyses could be performed. 

A Prev.iew of Plans for the Future Evaluation 

This report represents a transition point in the PDC 
evaluation. The conclusions from the feasibilicy study were 
accepted by the Administration for Children, Youth and Families 
(ACYF) and the longitudinal phase of the evaluation was launched 
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the status of the sample as the study entexs its lonaitudinal 
phase. This is also a transition, pQintp in terms, of the 
richness of information, ava^able about the various partici-- 
pants in PDC^. At 'the Head Start and kindergarten l^^elsr our 
data collectioii for the impact study was limited to data on 
child oixtcomes. Beginning in 1979, in the spring^ of the 
chiiaten' s first grade year, a more comprehensive data collection 
began, with parent interviews, teacher interviews , classroom 
observations, and adminis^trator interviews -added to th^ child 
measures . . . . 

The findings reported here raise more questions than they 
answer. We can describe the lack of clear evidence of the , 
program's impact on those characteristics ■ of children ' s develop- 
" ment and achievement that are measured by the 'various tests, 
observations ajid .ratings We cannot, however, offer much in 
the way of satisfactory explanation for these findings. We 
do not know, for example, the extent to which PDC teachers 
really implemented an individualized instructional program, 
or the extent to which. thev differed from the comparison group 
teachers on this and other* factors x. We do not know the exrent 
to which PDC parents participated in classroom activities 
with their children, or whether PDC parents did more of this 
than parents of ciomoarison group children. The next report 
in this series, focusing on spring 19 79 data, should provide 
answers to questions that seek explanations for the impact 
findings. It is our hope that the present report is useful 
in raising the appropriate questions for exploration as oUr 
analyses of the more comprehensive data begin. 

This report follows the pattern established \n preceding 
impact reports. Chapter II documents the methods of data 
collection and summarized the analysis questions addressed 
in the remainder of the report. Chapter III presents the 
results of analvses designed to help us understand the nature 
of the sample and the characteristics of the various instruments. 
It includes a presentation of descriptive data on the sample 
and an analysis of attrition patterns and the effects of 
attrition on the characteristics of the sample. Chapter III 
then presents information on the instruments, in terms of the 
distributions of scores, internal consistency and stability 
of inu^vidual itieasures, the factor structure of the battery, 
and the relationships of the measures to criteria of "social ^ 
competence." Chapter IV examines treatment effects, presenting 
findings of program impact separately for the English-dominant 
and Spanish-dominant samples. The entire report is summarized 
in Chapter V. 
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Data Collection Procedures 

T?o establish a data ^ll^^ction routine thiat would result VI 
in data of the hiahest possibly quality , the procedures followed^ 
in the four precedina data collection peri6d's were" continued - 
with minor modifications: . , . ^ ; 

• An organizational structuife for individuals ' t 
involved in *the datA collection effort -was 
outlined, role responsibilities were ' defined , 

and a detailed training manual was procured. 

• A traini-ng model- was designed that specified' 
tester performance standards and provided for 
a four-day tester training session with large- 
group, small-group and individualized instruc- 
tion, daily reviews of each tester's perfor- 
mance, and discussions of potential problems. 

• Onsite monitoring of testers by trainers was 
conducted prior to the start of the actual 
testing. 

• During the data collection period, testers were 
responsible for monitoring each other's perfor- 
mance on a weekly basis. 

• Site coordinators collected completed data 
each week and checked it for obvious errors or 
omissions before sending it to the High/Scope 
Fo\indation . 

Each of these procedures is discussed below. 
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Field Organization ^ - ' 

The roles of the personnel who conducted, fiel^d data 
collection were explicitly defined in the Field Procedures 
Manual in order to systematize responsibilities. For example, 
site coordinator responsibilities included contacting the 
PDC coordinator regarding the start of testing; setting up 
and chairing a meeting with the kindergarten teachers involved 
i^ the evaluation', or contacting them individually; keeping 
in contact with High/Scope's supervisor of field operations 
about the status of data collection and any problems that 
the site was having; checking all completed data on a v/eekly 
basis; keeping up-tb-date records on the status of the data 
collection; carrying out any needed training; tes-^ing (and, 
in some cases, obs'erving) children; and ^monitoring testers- 

Training Model 

Training sessions for both trainers and. testers were . 
held in early April 1978. Since all but one of the seven 
' trainers had been involved in 'previous PDC training sessions, 
a brief, one-and-on,e-half -day session was held to review and 
practice the child measures. More time was then spent with 
the new trainer on a one- tb-one' basis ^ In addition, the 
three observer tr-aineirs met for one day to review, the obser- 
vation system and cpde videotapes.' The, ten observers spent 
two days focusing on the PDC Classroom Observation .System 
before being joined by the ,17 testers for four days of training 
in the child measures . 

Tester training . During the onsite tester training 
sessions, each test was reviewed and practiced. Practice 
sessions involved the use of tfest "scripts." The scripts 
consisted of test instructions, child responses, and rationales 
for scoring. In' using the scripts, two testers would pair up 
and one, the "child," would perforra as indicated on the script 
while the other tester administered the test without the 
script. This provided an excellent learning situation since 
the child respona^es included on the script covered all the 
administration rules and gave the testers a chance to work 
with and correct each other. 



^The spring data collection began the week of April 17 at all 
sites except Florida. The length of the data collection period 
was fairly constant across sites, with most testers finishing 
within se.ven or eight weeks. In Florida, testing and classroom 

■observations were conducted , over a three week period by 
High/Scopp. staff. 
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Since it is critical that testers administer the tests 
in a standard way, each tester was .systematically "checked- 
out" on all of the child measures before the- end of the training 
session. During this procedure a trainer played the role 
of the child (also recording, the "child's" responses) while 
a tester administered one or more of the child measures to 
her. The trainer (acting as the child) responded in standard 
ways to each item on each test in order to insure that (1) 
aach tester was exposed tb the sarnie situations, and (2) the 
trainer could assess the tester ' s" handling of critical child 
responses. For example, on the PIPS interview, there are 
specific things for the tester to say if the child gives an 
unrelated answer, a repeated answer, refuses to answer, and 
so on. By exhibiting all these behaviorNS in the check-out ^ 
sit'.ation, trainers were able to assess the tester's under^ 
standing and ex^Dertise in administering each of the child 
measures. * 

Standards were set foif^acceptable performance during the 
tester "check-buts, "# and if these standards could not be mer, 
additional training and practice was prescribed. Check--outs 
were theA repeated at a later time during the training session 
to insure correct test administration. 

Observation training . As mentioned, the two days preceding 
the'tester training session were devoted to observer training . 
In previous training sessions one tester from each site had 
been identified and trained as the PDC cla^ssroom observer. 
Thus, nine Of the ten interviewers were familiar with the 
observation system. During this time the trainers reviewed 
the entire ^^§tem, paying particular attention to those 
categories or-items that the observers had liad problems coding 
during the previous data collection period. The observers and 
trainers sptent many hours coding videotapes and discussing the 
correct codes. The final training activity involved the 
coding of the observation reliability tape. 



Monitoring / . • 

Onsite monitoring . The onsite monitoring occurred the 
week following the- training session. At this time each of the. 
newly ,hired testers was required to administer the PDC measures 
to a child while the trainer observed. After watching a tester, 
the trainer "provided any additional feedback to the tester 
that was necessary; for improving her interactions with the 
children. This^ procedure served two purposes: it gave the 
'trainer an indication of how well the new testers could 



establish rapport 'and interact with children, and it helped 
alleviate some of the anxieties the inexperienced testers 
felt about administering the measures to children. 

Week ly monitoring . During the course of a testing week, 
testers alternately monitored each other; the one acting as 
monitor simultaneously completed the test booklets and the 
individual monitoring forms for each test. ' After the session, 
the monitor and tester discussed any errors, and the monitoring 
booklets and forms were sent to the supervisor of field 
operations at the High/Scope Foundation to be reviewed. 

Weekly Pre-Transmittal Data Checks 

Testers were required to give or send their completed 
data to the site coordinator at the end of each week. The 
site coordinator th,en checked these tests, plus any she had 
completed, for recording/scoring errors. (Site coordinators 
and interviewers had reviewed a checklist specifying what to 
look for when reviewing each completed booklet , e .g. , "Is the 
identification complete?" "Did the interviewer fail to give 
a second trial when it should have been given?" "Did the 
interviewer skip an item?") Errors were pointed out to the 
particular tester and,- if necessary, further training was 
provided" by the site coordinator. The site coordinator also 
kept track of all completed data (in addition,.to the individual 
records each tester kept of her classes) and mailed the 
completed data to the High/Scope Foundation on a weekly basis. 

Recording and Scoring of Data 

In addition to the site coordinators' pre-submittal 
check, data collected by the testers were also checked by 
the supervisor of field operations at the High/Scope Foundation 
Errors in rec6rding or. coding were identified and explained 
to the site Coordinator, who then discussed them with the 
other testers. 

Once the raw data had been screened for accuracy at 
High/Scope, they were sent to the data processing section 
to be tagged with unique identification numbers for each 
student", scored and verified, then keypunched and verified. 



Data Collection Sequence 



Once the evaluation sample children were located in the 
district schools the testers <;iivided the classes among them- 
selves. In making these divisions two factors were taken 
into account: (1) the order in which the classes were to be 
completed was such that testers would be collecting data 
simultaneously in the PDC and comparison schools ^ and (2) 
each tester would be testing both PDC and comparison children r 
thus eliminating the possibility of tester bias for either 
group- 

Determining Child's Language Capabilities 

The procedure followed by testers in determining the 
language capabilities *of„ children in the testing sample 
was (1) to ask ^t^he child's classroom teacher for his or her 
judgment, (2) to observe the child's verbal behavior. in- 
natural classroom conditions, and (3) on the basis of these 
indications, to administer the English or Spanisji version, 
or both, of the Bilingual Syntax Measure (BSM) . In most 
cases, this screening process produced consistent conclusions, 
and subsequent testing was accordingly . conducted in English 
or Spanish or both.. (Tn some cases this screening process 
led to the conclusion that a child was proficient in some 
third language, but not English or Spanish; these children 
were excluded from the testing sample.) When the screening 
process proved inconclusive, the tester carefully weighed 
all available information to reach, a conclusion about the 
child's language capabilities. 



Data Analysis Procedures 

c 

Chapters III and IV of this report present the results of 
four stages of analysis, focusing on: 

• descriptive characteristics of the PDC and comparison 
samples 

c attrition patterns within the samples 

characteristics of the instruments in the spring 
1978 PDC battery 



• effects of the PDC program on participating children. 
The procedures for these analyses are described briefly below. 



Descriptive Characteristics of ±he Samples 

In order to characterize the composition of the PDC and 
comparison samples ^descriptive statistics were c6mputed for 
children in these samples at each site and at all sites combined. 
These statistics are based on the full sample of children 
tested r but subsequent analyses were carried out on an analytic 
::^ubsample that excluded children who have handicaps likely 
to impair test performance or whose dominant language is not 
that of their local testing sample. 



Attrition Patterns 

Representativeness of remaining samples . Children who 
have departed the PDC and comparison samples since fall of 
their Head Start year were compared with t^e children remaining 
in the respective samples on a number of background variables 
and orf^ f all iTlS test scores. .The purpose of this phase of 
analysis was to determine whether any selection effect may 
be operating that diminishes the representativeness of the 
samples remaining. The hypothesis of bias due to attrition 
was evaluated by means of t tests .and chi-square analyses. 

Comparability of remaining samples . The remaining PDC 
and comparison samples were compared in the same way. on the 
same dimensions to determine whether they may still be con- 
sidered equivalent in terms of their characteristics at the 
time of program entry (fall 1976) . 



Characteristics of the Instruments 

Because all the instruments in the battery have , in 
earlier PDC analyses , passec^^ through at least two screenings 
based on psychometric criteria-, no further screening was 
considered necessary before the data were entered into the 
analyses reported here. The psychometric analyses presented 
in this report are included mainly for purposes of instrument 
definition and documentation. The psychometric properties 
reported, on the basis of past and present analyses, include: 

• means and standard deviations 

• reliability (internal consistency) 



m validity 

• stabili^ty 

• sensitivity to change 

• intercorrelations 

• factor structure 

• relevance to "social competence.", 

Examination of Treatment Effects 

In examining PDC ' s effects on children, three analytic 
questions were posed: ' , 

1. Has a group difference [between PDC and comparison 
children] arisen since fall of the Head Start year? 

2. Has a group difference arisen since spring of the 
Head Start yecrt? 

3. Is there a difference in the growth curves, of the 
two groups? 

These' questions were addressed by means of analysis of covariance, 
and repeated' measures analysis of covariance . 

In separate analyses, PDC ' s effects at the classroom 
level (as distinguished from the child level) were examined 
on. dimensions measured by the original version of the PDC 
Classroom Observation System. This version, now superseded 
by a new system, was used for the last time in spring 1978. 
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CHARACTERISTICS OF THE SAMPLE AND OF THE INSTRUMENTS 



Characteristics of the Sample 



Gener al Description 

— — ■ — 0 

A total of 782 children were tested at eleven PDC sites 
in spring 1978. Table 1 shows the niimber of children in the 
PDC and comparison groups at each site and describes the 
composition of each group - 

Not all of the children tested entered into the analyses 
presented in this report; the actual number of children 
included in analyses is shown in the far right-hand column 
of the table- Excluded from the analysis were children 
whose dominant language whs other than English (except for 
the Spanish-dominant children in California and- Texas, who 
comprise their own analytic groups) ; also excluded were children 
with handicaps likely to impair test performance unduly - 

Attrition Patterns 

As can be seen in Table 2, the PDC testing sample has been 
reduced by 42% since fall 1976, the comparison testing sample 
by 29%. Overall, this represents an attrition rate of 36%. 
In the first phase of the PDC evaluation we had projected 
attrition rates of 37% for the PDC group, 42% for the comparison 
group, and 39% for the two groups combined. Table 3 shows 
that movement from the school area was the major reason for 
sample attrition. The. differential attrition of PDC and 
comparison children is due to the fact that we continued to 
test comparison group children who moved from one local school 
to another as long as the new school did not house a PDC program 
When PDC children moved, however, there were few or no other 
PDC schools to move to. Thus we did not continue testing 
these children, since they were no longer in a PDC program. 

In some cases, children who are shown as having left the 
sample betweqn fall 1976 and spring 1977 returned. again m 
spring 19 78- This occurred when children who left the^^Head 
Start program before 1977 testing were located in the "correct' 
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Table 1 



Descriptive Characteristics of the Sample 
Spring 1978 





Nuinber in 
Full Sample 


% Handicapped 


%'Black 


% Hispanic 


7o American Indian/ 
Native Alaskan 


% White 


% Asian/Pacific 
Islander 


CALIFORNIA-English ^°^p 


2^ 

17 


0 
6 


5 
6 


90 
59 


0 
0 


5 
35 


0 
0 


pnr 

CALIFORNIA-Spanish ^^jjp 


4 

10 


0 
0 


0 
0 


"100 
100 


0 
0 


0 
0 


0 
0 


COLORADO ■ 11^^ 


25 
22 


' 12 
9 


12 
0 


76 
64 


0 
0 


12 

36 


0 
0 


pnr 

CONNECTICUT 


32 
46 


3 
2 


50 
83 


25 
11 


3 
0 


22 
7 


0 
0 




40 
34 


5 
0 


100 
91 


0 
6 


0 . 
0 


0 
3 


0 
0 


GEORGIA PDC 


30 


3 


80 


0 


0 


20 


0 




^5 
22 


4 

0 


44 
14 


4 

0 


0 
4 


52 
82 


0. 

0' 


MARYLAND ^°^p - 


32 
30 


25 
23 


47 
37 


9 
23 


0 
0 


41 

30 


3 
10 


MICHIGAN ■ 


31 
51 


13 
8 


61 
75 


3 
2 


0 
0 


36 
24 


0 
0 


TEXAS-English 


29 
23 


0 

0 


3 
0 


55 
87 


0 
0 


41 
13 


0 
0 


pnr 

TEXAS-Spanish ^omp 


. 17 
19 


0 
16 


0 
0 


100 
100- 


0 

0- 


0 
0 


0 
0 


UTAH III, , 


28 
60 


18 
10 


0 
7 


^1 
12 


0 
7 


79 
75 


0 
0 


WASHINGTON [f^^ 


35 
47 


17 
13 


14 
45 


0 
2 


17 
2 


51 
49 


17 
2 


WEST VIRGINIA [^^^ 


26 
26 


8 
19 


4 

12 


0 
0 


0 
0 


96 
88 


0 
0 


TOTALS BY GROUP ^^^^ 


375 
407 


9 
9 


36 

37' 


^5 
24 


2' 

1 


35 
37 


2 

1 


TOTALS, ALL GROUPS COMBINED 


782 


9 


37 


24 


2 


36 


1 



ETHNICITY 



SEX 


% Female 


to 

^ 


81 
65 


19 
35 


50 
60 


50 
40 


40 
50 


60 
50 


53 
41 


47 
59 


35 
47 


65 
53 


57 


43 


52 

50 


4S 
50 


44 
47 


56 
53 


, 55 
55 


45 
45 


45 
43 


55 
57 


35 
68 


65 
32 


46 
50 


.54 
50 


51 
60 


49 
40 


54 
69 


'46 
31 


4$ 
53 


51 

^7 


51 


49 



"LAN 




GUAGE'' 


SI 


J= 

(/I 


cn 


(~ 

ta 

It 


uu 








lOO 


0 


100 


0 


0 


m 


0 


100 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


TOO 


0 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


0 


100 


0 


100 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


100 


0 


94 


6 


93 


7 


94 


6 



jQ 
J« (O 

Q) >^ 
jQ I— 
E to 



TT 
16 
~T 

10 

20 



24 
44 



38 
34 



29 
22 



24 
23 



27 
47 



18 
11 



17 
16 



23 
54 



29 
41 



24 
21 

359 



683 



In kindergarten year. 

'^Children who switched language groups from the Head Start to the kindergarten 
year were eliminated from the analytic sample, as were children with serious 
handicaps, based on Head Start year data. 
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Table 2 

Summary of Year-to-Year Attrition^ 





PDC 






COMPARISON 






Fall Spring ! 
1976 1977 
N N 


Spring 

1 mo 
1 y /o 

N 


1976- 

1 Q77 

% Drop 


1977- 

1 Q7R 

% Drop 


1976- 

1 Q7R 
1 y / 0 

% Drop^ 


ra 1 1 
1976 
N 


C n "J n H 

opri riy 
1977 
N 


Spring 
1978 
N 


1976- 
1977 
% Drop 


1977- 

1978 
% Drop 


1976- 

1978 
% Drop 


Cal i f nrni a-Enal i sh 

III \J 1 illU ^11 1 • w 1 1 

Cal ifornia-Spanish 


Of 


J 1 


21 


16 


32 


43 


25 


21 


17 


16 


19 


32 


7 


U 


4 


14 


33 


43 


15 


11 


10 


27 


10 


33 


Colorado 




DU 




g 


50 


55 


32 


25 


22 


22 


12 


31 


Connecticut 


iJ u 


ij ^ 


32 


5 


40 


43 


57 


54 


46 


5 


17 


19 


Florida 


47 


39 




17 




1 5 


39 


35 


14 


10 


3 


13 


Georaia 


46 


43 


30 


7 


30 


35 






No aompariaon 8a 






Iowa 


50 


43 


25 


14 


42 


50 


54^ 


48 


22 


11 


54 


59 


Maryland 


44 


41 


32 


7 


22 


27 


58 


47 


30 


19 


36 




Michigan 


66 


58 


31 


12 


47 


53 


64 


58 


51 


9 


12 


20 


Texas-Engl isn 


26 


23 


29b 


12 






20 


19 


23^ 


5 






Texas-Spanish 


38 


31 


17^ 


18 


45 


55 


17 


36 




3 


47 


49 


Utah 


68 


64 


28 


6 


56 


59 


'61 


.54 


60 


12 


_c 


2 


Washington 


58 


52 


35 


10 


33 


40 


76 


61 


47 


20 


23 


38 


West Virginia 


46 


38 


26 


17 


32 


44 


37 ^ 


34 


26 


8 


24' 


30 


TOTAL 


644 


572 


375 


11 


34 


42 


575 


503 


407 


13 


19 


29 



^Figures refer to number of children in the full sample, as distinguished from the reduced analytic sample. 

^The language dominance classification (English/Spanish) of 24 children changed from the Head Start to the 

kindergarten year. Thus the composition of the English- and Spanish-dominant groups in California and lexas 
changed somewhat, 

CRcdcsinnation of group monborship in Utnh resulted in an increase in sample size from 1977 to 1-978 (see text 
gj^^ for explanation). » 2 " 



Table 3 

SuTTinary of Reasons for Attrition 



Number of Children Not Tested Due To: 





Moving, 
Withdrawal 


Continual 
Absence 


Testing 
Refusal 


Change in 
Group 
Designation 


Other 
(including no •• 
permission slip) 


1977 1978 


1977 1978 


1977 1978 


1977 1978 


1977 1978 


PDC 


63 " 176 


2 0 


2 0 


0 27 


4 16 


COMPARISON 


67 78 


2 0 


0 0 


0 0 


4 56 


TOTAL 


130 254 


4 0 


2 0 


0 27 


8 72 



0 - 



elementary school in 1978., There we're 60 children in all 
who, in this way, left "the sample and theii returned. All- of 
them had experienced at least three months of the Head Start 
program before "departureT^ 

Another situation affecting apparent attrition arose in 
Connecticut and Utah. In these sites ^ PDC and comparison 
children attended the same Head Start center and were desig- 
nated "PDC" or "comparison" depending on the elementary school 
thay were likely to attend. However/ some of the^ "PDC" 
children went on to comparison schools and vice-versa. Since 
there had been no fundamental difference in the Head Start 
experiences of these two grouf^s in either site, children who 
moved on to PDC and comparison* schools were classified as 
members of those respective samples regardless of their original 
designation. These were the only instances in which children 
who switched groups were retained in the study. 

A final note on attrition and its implications: although 
the attrition rate is somewhat lower than projected overall r 
among PDC children it is higher than we had estimated (42% 
instead of 37%) . Added to this is the fact that an entire 
site, West Virginia,' ceased participating in the program 
after the spring 1978 data were collected. Thus the number 
of PDC children likely to remain in the sample through the 
end of grade 3' is now almost certainly smaller than our earlier 
(March 1977) projection of 258. Our estimates at this point 
suggest a final PDC sample size of about 22 0 — an average of 
about 20 'PDC children* per site. The size of the comparison 
sample at the end of grade 3 is likely to be larger than this, 
given current trends. In our judgment, these reduced samples 
will still be adequate for purposes of the evaluation. 

Attrition's effects on group comparability . Attrition, 
beyond reducing sample siz^, can alsp have the effect of 
altering the group equivalence so important to long-term 
comparative studies like PDC. To examine this possibility, 
we conducted a series of analyses to assess the differences 
between the remaining PDC and comparison samples on major 
background characteristics and on performance measures obtained 
at the time of the children's entry into Head Start. The 
results of these analyses (chi-square tests for categorical 
variables, t tests for metric variables) are shown in Table 4 
for the English-dominant analytic sample. On only two of 
the dimensions examined, POCL-1 and POCL-2, do the PDC and 
comparison groups differ significantly, and these differences 
are rather small (.14 standard deviations in the first case, 
.25 in the second) . Furthermore, when all fall 1976 test 
scores are taken jointly as dependent variables in a multivariate 
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Table 4 



Comparability of Remaining PDC and Comparison Children 
on Baseline (Fall 1976) Characteristics * . 

(English^Dominant AnaUjtic Scrnple) 



^ — - - 


1 

PDC Samel e 

Svvir.a :. "3 


Comparison Sample 

S::rinc: 1373 


*" »■» 


SOS 




zzz 








Ethnicitv i%) 




42 


Black 


42 


Hispanic 


17 


15 


American Indian/Native American 


2 


2 


White 


37 


40 


Act an /Par 1 "f "i r T^landpr 


2 


1 






48 


Male 


47 


Female 






rrior rrebLnuui v^^^y 




•18 


Yes 


14 


No 


86 


82 


Age (months) 


53.67 


53.79 


Number of Siblings 


1 .96 


1 . 80 


Mother's Education 


* 

11 .02 


11.13 


TeA-t Sco/ieA { ¥clU 1976] 






BSM-English 


9.28 


9.47 


WPPSI 


4.84 


5.07 


Verbal Fluency 


6.41 


6.41 


Verbal Memory-1 


12.99 


13.68 


Verbal Memory-3 


2.60 


2.76 


Arm Coordination 


3,43 


3.61 


Draw-A-Child 


3.95 


3.96 


PIPS 


2.03 


2.15 


POCL-1: "Task Orientation" 


32.43 


34.00* 


POa-2: "Sociability" 


12.53 




13.47* 



*PDC-comparison group difference. on this' variable significant with p < .10 (two-tailed). 

Note: The PDC-comparison difference on all test scores taken jointly (MANQVA) is 
not statistically significant (p > .10), , 
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analysis of variance, the difference between the profiles of 
the two groups does not approach statistical significance. 
And as Table 5 shows, the results of the same analyses for 
the Spanish-dominant analytic sample were equally positive 
(though It must \e noted that the small size of the Spanish- 
dominant sample does not provide high power to detect group 
differences, should they exist). 

All in all, the match that was accomplished between PDC 
and comparison groups in fall 1976 has withstood the effects 
of attrition, and group comparability remains excellent, at 
least for the English-dominant analytic sample. As for the 
Spanish-dominant analytic sample, our results suggest the same 
conclusion but we cannot state that conclusion with the same 
confidence, due to the low power of inference we command with 
that sample. 

■•.At trition's effects on group representativeness . Attri- 
tion, whether it alters group comparability or not, can 
potentially affect the representativeness of the remaining 
groups. For example, if families of higher socioeconomic status 
were more mobile, then their children would be likelier to 
leave the PDC and comparison samples. If the trend were equal 
in both groups, thic change in representation would not be 
revealed by analyses of the comparability of the present g^o^PS- 
Thus, to examine this additional contingency, we contrasted the 
children who have remained in the analytic samples through 
spring 1978 with those who have left, using the same 
dependent measures used in analyses of copiparability . . The 
, results of these analyses are displayed in Tables 6 and 7. 
On the basis of the evidence given, we can conclude once again 
that attrition has not biased the remaining samples: for the 
PDC and comparison groups alike in the English- and Spanish- 
dominant samples, there are very few , statistically significant 
differences on the dependent measures, taken one at a time. 
And when the dependent measures %re examined jointly in multi- 
variate analyses of variance, there are no indications whatever 
of significant differences between. the profiles of the remaining 
and departing groups. This conclusion holds for all four series 
of analyses performed: on the English-dominant PDC group, 
the English-dominant comparison group, the Spanish-dominant PDC 
group, and the Spanish-dominant comparison group. 

^t this point, then, we can be confident in assuming that 
the remaining samples are representative of the original samples 
established in fall 1976. Thus conclusions based on the 
experience of the present samples can be -generalized to the 
Head Start population from which the PDC and comparison group 
children 'were drawn. 
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Table 5 . 

Comparability of Remaining PDC and Comparison Children 
on Baseline (Fall 1976) Characteristics 

(Spanish-Dominait Analytic: Sample) 





PDC Sample 1 

' Ewzyij 13? 3 i 


Comparison Sample i 


N (approxirrjjte) . . ^ 


61 


Ct\J 


Backgnvund ChahJicXQJii^tic^ 




\ 

0 

100 
Q 
0 
n 


Ethmcity \/o) 

Black 
* Hispanic 
American Indian/Native American 
White 

Asian/Pacific Islander 


,0 

■ IQQ 
0 
0 
u 


Sex (%) 
Male . ' 
Female 


62 
38 

• 


39 
62 


Prior Preschool {%) 
Yes 
No 


14 

86 


35 
65 


Age (months) 


54.81 


55.65 


Number of Siblings 


2.38 


3.08 


Mother's Education 


6.41 


5.57 


TeJ^t Sco^^s [TM 1976) 


• 




BSM-English 


.17 


2". 2 5 


BSM-Spanish 


12.75 


12.50 


WPPSI 


8.71 


7.Q0 


Verbal Fluency 


4.40 


5.00 


Verbal. Memory-1 


13.43 


12.77 


Verbal Memory-3 




L . DO 


Arm Coordination 


4.24 


4.46 


Draw-A-Child 


4.25 


4.69 " 


PIPS 


1.95 


1.92 


- POCL-1: "Task Orientation" 


30.10 


31.35 


POCL-2: "Sociability" 


11.33 


12.42 



Note: The PDC-comparison difference on all variables singly and on all test scores 
jointly (MANOVA) is nonsignificant (p > .10). 
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Representativeness of Remaining PDC and Comparison Children 

with Respect to Original (Fall l'976) Groups 

(Englieh-Dominaint Analytio Sample) 





PDC 


f 


Comparison 




Children Remaining | 


vCh^ldren Departing 


- 

Chi Idpen Remaining 


Children Departing 




in Testing Sample^ 


from Testing Sample 


in Testing Sample 

i 


from Testing Sample 

1 ■ 




303 


205 


1 


160 . 












Ethnicity {%) ' 


... 




• 

42 




Black 


42 


35 


34* 


Hispanic 




. 22 


15 


24 


Minencan ina idn/rtat i ve 


2 




2 


.3 


Ampri can 










Whi tp 

nil 1 


37' 


40 


/in 


00 


Asian/Pacific Islander 


2 • 


2 


1 


3 


Sex (%) 








47 


Male 


47 


53 


48 


Female 






CO -. 


53 


rrior rrescnoo 1 \/o) 










Yes 


14 f • 


12 


18 


11* 


No 


86 


• 88 


82 


89 


Age (mogths) 


53.68 


' 53.43 


53.79 


53.68 


Number of Siblings 


» 1 .96 


1.77 


1.80 


1.99 


Mother's Education 


11.02 


11.15 


11.13 


11.14 


toAt 'Scofia> [VaJUi 1976) ' 










BSM-English 


9.28 


9.40 ■ 


^ 9.47 


9.58. 


WPPSI ■ 


4.84 • 


5.11, 


5.07- * 


5.11 


Verbal Fluency 


6.41 


5.39* 


6,41 


6.25 



♦Difference on this variable between remaining and departing groups significant with p < .10 (two-tailed). 

' o ^ . 2 / ' ' ■ , • • J 

ERIC . , , — . 



Table 6 
(continued) 



Verbal Memory- 1 
Verbal Memory- 3 
Arm Coordination 
Draw-A-Child 
PIPS 

POCL-1: "Task Orientation" 
POCL-2: "Sociability" 




Children Remaining 
in Testing Sample 



12.99 
2.60 
3.43 
3.95 
2.03 
32.43 
12.53 



Children Departing 
from Testing Sample 



12.19 
2.70 
3.46 
3.88 
1 .89 
32.89 
12.76 



13.68 
2.76.. 
3.61 
3.96 
2.15 
34.00 
13.47 



from Testing Sample 



13.90 
2. 54 
3.14 
4.30 
2.05 
32.35* 
12.46* 
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Table 7 



Represeita(tiveness of Remaining PDC and Comparison Children 
with Respect to Original (Fall 1976) Groups 

(Spanish-Dominant Analytia Sample) 





PDC 


Comparison 




Children Remaining j Children Departing 
in Testing Sample {from Testing Sample 


Children Remaining | 
in Testing Sample | 


Children Departing 
from Testing Sample 


N (approximate) 






26 j 


11 


J — 


j 






■ 


ttnmcity \%) 


j 


0 


0 j 




Black 


0 1 


0 


Hispanic 


100 } 


100 


100 } 


100 


American Indian/Native 


0 j 


n 
U 


0 1 


0 


American 




0 


White 


Q j 


U 


0 1 


Asian/Pacific Islander 




0 


n 1 

1 


0 


Sex (%) 


1 




39 j 


36 


Male 


62 j 


50 


Female 


38 1 


bO 


61 j 


64 


Prior Preschool {%) 








55 


Yes 


14 j 


15 


No 


86 1 


85 


65 1 


. 45 


Aqp (months) 


54.81 j 


55.08 


55.65 


55.82 


Niimhpr of Sihlina^ 


2.38 i 


1 .33* 


3.08 


2.18 


Mothpr*<; Fduration 


6.41 i 


7.11 


5.57 


6.55 


Tc>\t ^on/iPA (Foii. J 976 1 










BSM-English 


.17 ! 


3.00* 


? 25 


4.83 


BSM-Spanish 


12.75 j 


13.18- 


12.50 


10.36 


WPPSI 


8,71 ! 


7.23 


7.00 


5.55 


Verbal Fluency 


4.40 j 


3.00 


5.00 


5.64 


Verbal Memory- 1 


13.43 i 


13.00 


12.77 


13.00 


Verbal Memory-3 


2.25 j' 


2.23 


2.58 


1 .82 



^ *Difference on this variable between remaining and departing groups significant with p < .10 (two-tailed). 
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Table 7 
(continued) 





• PDC 


Comparison 




Children Remaining i 


Children Departing 


Children Remaining 


{Children Departing 




in Testing Sample 


from Testing Sample 


in Testing Sample 


j from Testing Sample 


Arm Coordination 


4.24 


4.13 


4.46 


1 3.03 


Draw-A-Child 


4.25 


4.25 


4.69 


1 4.45 


PIPS 


1 .95 


1.67 


1 .92 


1 2.09 


POCL-1: "Task Orientation" 


3o:io 


26.62 


31 .35 


1 30.18 


POCL-2: "Sociability" 


11.33 


10.15 


, 12.42 


j 12.55 

1 ' 



*Differerce on this variable between remaining and departing groups significant with p < .10 (two-tailed). 

Note- The difference between remaining and departing children on all test scores taken jointly (MANOVA) 
is nonsignificant (p > .10). This is true for both the PDC and comparison groups. 
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Characteristics of the Child Measures 



Review of Past Findings 

One of the major tasks 'of Phase 1 of the PDC evaluation 
was to examine the adequacy of each of the instruments in 
the PDC battery. Criteria were set regarding critical measure- 
ment properties, and the suitability of each instrument was 
judged against its showing on these criteria. On this basis, 
a number of instruments were deleted from the' batrery or 
revised. The instruments that were retained constituted a 
battery that, in our judgment, satisfied most or all of the 
conditions that could reasonably be applied. A summary of. 
past psychometric findings is given in Table 8. Descriptions 
of the instruments themselves appear in Appendix A. 



Revision of the PDC Child Rating Scale 

The Child Rating Scale (CRS) , as administered in 1976, 
1977, and 1978, has consisted of 39 items, all revolving around 
the child's behavior as observed by the classroom teacher. 
Before, proceeding with this year's analysis of CRS data, we 
\indertook to determine whether it would be possible in the 
future to reduce the number of items while retaining the 
integrity of the scale, thereby reducing the rating burden 
on the teacher without diminishing the measure's utility to 
the evaluation. The analyses aimed at reducing the CRS pro- 
ceeded in several steps, described. below. 

1. In the past, factor analysis of the 39-item CRS 

has consistently yielded six fairly stable factors. 
Analyses of PDC-comparison differences on the CRS 
have been based upon scale scores formed by summing 
each child's ratings on the items associated with 
each of the six factors. The first step in our 
reduction procedure was to determine which of the 
items in each of these scales were the best pre- 
dictors of the total scale score. We began with 
CRS data from spring 1977 , allowing for independent 
replication of the results later with spring 1978 
data. Using regression techniques, we selected 
the smallest set of items that predicted each of 
the six factor scores with at least 90% precision^ 
(R^ > .90). These sets, which turned out to consist 
of only 17 items in all, represented reduced versions 
of the six empirically determined scales. 
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Table 8 

Sunmary of the Psychometric Characteristics of the Battery in Fall 1976 and Spring 1977 

(Based on Data for Children in the Analytic Sample) 



MEASURE 



EWGLISH-P0MIWAW7 CHUPRtW 

BSM-Enqlish 
BSrt-Spanisn 
Verbal Fluency 
Verbal Memory-1 
Verbal Memory -3 
Draw-A-Child 
Arm Cdordi nation 
PiPS 

POa-1 : "Task Orientation" 
POCL-^: "Socidbn ity" 
CRS-1 : Friendliness" 
CRS-2: "Aggressiveness" 
CRS-3: "Perseverance" 
CRS-4: Independence" 
CRS.5: "Self -Assurance" 
CRS-6; "Resourcefulness" 

SPAW/SH-POMIWAWT CHUPREW 

BSM-English 
BSM-Spanish 
Verbal Fluency 
Verbal M,emory-l 
Verbal Hemory-B 
Oraw-A-Child 
Arm Coordination 
PIPS 

POCL-l: "Task Orientation" 
POCL-2: "Sociability" 



internal Consistency 
( nbach's Alpha) 



Fall 1976 



.84 
.95 
.76 
.85 
.82 
.84 
.65 
a 

.95 
.90 

c 

c 

c 

c 

c 

c 



.93 
.86 
.81 
.89 
.84 
.78 
.73 
a 

.96 
.96 



Spring 1977 



.83 
.94 
.76 
.82 
.81 
.81 
.69 
a 

.95 
.87 
.85 
.84 
.89 
.77 
.91 
.74 



.90 
.70 
.76 
.85 
.78 
.69 
.71 
a 

.94 
.90 



Validity: Acceptable 
evidence? 



Stability 
(Fall-spring 
correlation) 



yes 

d 
yes 
yes 
yes 
yes 
yes 
yes 

d 

d 

d 
d 

d 
d 
d 



d 
yes 
yes 
yes 
yes 
yes 
yes 
yes 
yes 
yes 



.71 
d 

.49 
.51 
.47 
.54 
.43 
.38 
.44 
.43 

c 

c 

c 

c 

c 

c 



.90 
.59 
.52 
.69 
.39 
.39 
.24 
.17 
.39 
.39 



Sensitivity to 
change: Was actual 
spring mean > than 
predicted spring 
mean? 



no 

d 
yes 
yes 
yes 
yes 
yes 

d 

d 

d 

d 

d 

d 

d 

d 

d 



Factor Structure 



Fall 1976 



2 
d 
1 
1 
1 
2 
3 
I 
1 
1 
d 
d 
d 
d 
d 
d 



Spring 1977 



Relationship to social com- 
petence: % of variance ac- 
counted for by "social com- 
petence" criteria beyond 
background varibl es (s77) 



.07 
d 

.19 
.13 
.20 
.11 
.03 
.12 



^Consists of a single composite item, thus internal consistency coefficient cannot be computed. 
^^The subscales of the Child Rating Scale have been reconstituted since the analyses summarized here were done, 
^Not administered in the fall. 

d * 
Not analyzed. 

^These columns show which measures loaded on each of the factors (1, 2, 3, etc.) emerging from the various 
factor analyses. 

^Sinall sample size did not permit this analysis for the Spanish-dominant sample. 
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2. We then factor analyzed these 17 items to determine 
whether the same six-factor structure would emerge • 
Only three factors were actually found, but they 
bore a logical relationship to their "ancestor" 
factors that seemed to support the assumption 
that they adequately represented the original struc- 
ture (just as factor analysis reduces a complex 
data structure to a simpler pattern) . 

3. We calculated internal consistency (Cronbach's 
alpha) coefficients for each reduced scale and found 
that all coefficients exceeded .75, indicating adequate 
homogeneity of the scales' constituent items. 

4. Using scores based on the six scales of the full 39- 
item CRS we performed regression analyses to deter- 
mine the relationship between these six scores and 
children's scores on all other tests in the PDC 
battery. (These are the analyses we have presented 
in the past under the rubric "relationship to social 
competence.") Next, we performed parallel analyses 
using the three scale scores yielded by the reduced 
version of the CRS. We then compared the coefficients 
of determination (R^ values) obtained in these 
parall&l analyses to learn how much predictive pre- 
cision would be lost if we relied in the future 
on the reduced 17-item version 'rather than the original 
39-item version. Typically, precision was reduced 
by a margin of only 1 or 2% (e.g., from an value ^ 
of .19 to a value of .18 or .17). 

5. Finally, turning to spring 1978 dat^ , we assembled 
scores on the three reduced scales based on the 
17-item version. Factor analyses of these items 
replicated the factor structure found in spring 1977 
data. Internal consistency coefficients based on 
1978 data also paralleled those of the preceding year. 

" Having arrived at a versioji of the CRS that reduced the 
number of constituent items by more than half while sacrificing 
only slight predictive power, we decided to use the reduced 
version as .the basis .for analysis in this and future reports. 
Thus for this report we have constructed 1977 and 1978 CRS 
scores for each child that are based only on the 17 items 
that comprise tehe reduced version of the CRS, even though 
teachers completed the 39-item version in those years. From ■ 
spring 1979 through spring 1981, the CRS form that teachers 
complete will consist of the 17 items referred to here plus 
three more, hypothetically related to an "academic mo^.ivation 
factor. These were added to provide coverage of a dimension 
of behavior that was not explicitly represented on the original 
CRS. All 20 items of the new CRS are presented in Table 9. 



Table 9 

Item Clusters Produced by Factor Analyses 
of the Child Rating Scale^ 

Item No, on Revised 17- ...... „ 

Item CRS CRS-l : "Sel f -Assurance" 

1 Shows self-confidence 

5 Shows respect for or tolerance of 

' others' ideas and behavior or looks 

7 Is easily distracted when doing a task 

8 Attempts to solve social problems with 

little adult assistance 

10 Gets the attention of peers appropriately 

n Returns to unfinished tasks after 

interruption 

14 Cooperates and shares with others 

15 Recognizes others' feelings, responds 

appropriately 

15 Enjoys tasks he/she chooses 

Has a desire to master all kinds of 
skills 

19 Talks freely to children 



18 



CRS-2; "Aggressiveness" 

2 Shows verbal dislike or hostility* 

to others 

13 Uses words or wits to try to influence 

others 

17 ' Uses' physical forcfe to try .to control others 

Competes with others for toys, attention, 
achievement 



20 



CRS-3: "Dependence'^. 
o Is controlled or Influenced by others 

6 ■ . . Imitates others or follows them around 



^Three more items hav.e been added to 'tlpse' that emerged from empirical analysis of 
the original CRS, to represi^n't an "academic motivation" -factor: 

4. Is raotiyated toward academic performance 
9. Completes' assignments ' 
12.^ Is alert and interested in school work 

ERIC , , - : . 25 33 ' 



Psychometriq Analyses of Spring 1978 Data 

Score distributions . Table LO, gives the means and stan- 
dard deviations of each measure used in the*^ 1978 battery. 
These statistics are based ^upon data drawn from the samples 
of PDC and comparison chiLQren who have remained in the 
evaluation through spring^l978. (Some of these children were 
not tested on some measures in fall 1976 or spring 1977, even . 
though they were present, tjius the sample sizes for those 
years tend to be smaller than they are for 1978.) 

The year-to-year status of PDC and comparison children 
on these measures is represented graphically in Figures la 
to Ig (for the English-dominant sample) and 2a "to 2g ffor 
the Spanish-dominant sample). To allow comparison from measure 
to measure and from year to year, each of the gronp means 
graphed has been converted* to a s score that is based upon 
the overall mean and standard deviation for that measure in 
its first administration (fall 1976 for most of the measures; 
spring 1977 for the Child Rating Scale scores; the PIAT had 
never been administered before spring 1978).^ 

The graphs presented in Figures 1 and 2 allow examination 
of the relative overall status of PDC and comparison children 
at any timepoint on any measures in the battery. The graphs 
also depict changes over time in level of performance relative 
to earlier performance. Thus an "uphill" curvo indicates 



^The means plotted are standardized according to the formula: 



s' 

where 

■ i = treatment group (Pt)C/comparison) , 

j = assessment period (fall 1976, spring 1977, spring 1978), 
z = standardized mean , 
X = raw score mean, 

L'= raw score mean for PDC and comparison groups, combined, 



in baseline year, and 
S'= standard deviation for PDC and comparison groups, 
combinedv in baseline year. 
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. • , • ' Table 10 . 

* Means and Standard Deviations of Child Measures for Three Timepoints; 
Fall 1976, Spring 1977 and Spring 1978 (Based on Children Still Present in 1978) 



Measures 



Fall 1976 



N 



Mean SD 



Spring 1977 



N 



Mean 



SD 



Spring 1978 



Mean 



SD 



EhlGLlSH-VOhumT CHILVREN 

BSM-English 
BSM-Spanish 
Verbal Fluency 
Verbal Memory-1 
Verbal Memory-3 
Arm Coordination 
Draw-A-Child 
PIPS' 

PIAT-Math^ 
PIAT-Reading^ 

POCL-1 : nask Orientation" 
POCL-2: "Sociability" . 
CRS-T: "Self-Assurance"° 
CRS-2: "Aggressiveness" 
CRS-3: "Dependence"" 

spAMisH-vommhiT mmEhi 

BSM-English 
BSM-Spanish 
verbal Fluency 
Verbal Memory-l 
Verbal Memory-3 
Arm Coordination 
Draw-A-Child' 
PIAT-Math^ ... ~ 
RIAT-Reading^ 
PIPS 
POCL-1 
POCL-2 
CRS-1 



"Task Orientation' 
"Sociability." 



CRS- 
CRS- 



"Sel f-Assurance 
"Aggressiveness 
"Dependence"b 



..b 
„b 



635 
15 
626 
625 
621 
625 
626 
62? 



626 
626 



14 
46 
46 
47 
46 
46 
46 



9,:38 
6.40 
6. 41 
13.35 
2.68 
3.53 
3.96 
2.09 



33.26 
13.02 



1 .36 
12.61 
4.74 
13.06 
2.43 
4.37 
4.50 



4.27 
5.97 
r4.98 
6.47 
2.55 
3.04 
2.85 
1 .65 



9.40 
3.87 



3.41 
4.12 
4.37 
7.34 
2.46 
3.83 
2.28 



46 1.93 1.31 
4? 30.79 10. ■15' 
4?- 11.94 3.82 



595 
9 

596 
597 
592 
596 
597 
596 



565 
565 
524 
544 
546 



7 
42 
45 
45 
45 
45 
45 



45 
45 
45 
4 
4 
4 



10.26 
4.78 
9.34 

14.83 
3.87 
' 4.57 
5.69 
2.66 



34.37 
13.46 
3G.89 
11.62 
5.72 



4.57 
12.81 
7.42 
16.38 
4.18 
5.94 
6.44 



3.07 
37 .m 
13.40 

■38' 25 
10.25 
5.25 



4.01 
4.29 
5.73 
6.57 
2.74 
3.35 
2.67 
1.77 



9.74 
3.82 
6.95 
3.03 
1 .59 



3.74 
2.33 
4.35 
6.66 
2.67 
3.70 
1 .97 



1 .89 
8.71 
4.05 
4.79 
2.36 
0.96 



626 
19 
626 
■626 
626 
626 
626 
626 
615 
600 
595 
595 
552 
574 
572 



26 
46 
47 
47 
46 
47 
47' 
26 
26 
47 
42 
42 
41 
48 
45 



11 .93 
7.26 
14.57 
18.22 
5.75 
8.11 
7,63 
3.75 
13.40 
15.42 
36.84 
13.42 
35.18 
11 .53 
5.46 



8.50 
13.13 

9.47 
18.26 

5.11- 

9.25 

7.70 
11 .92 
10.54. 

4.00 
35.93 
13.17 
34.73 
10.94 

6.04 



3.09 
6.26 
5.60 
5.31 
2.54 
4.66 
1 .80 
1 .69 
3.96 
4.30 
8.57 
3.68 
7.71 
3.56 
1.84 



4.13 
2.45 
3.63 
6.57 
2.78 
4.74 
2.04 
3'r04 
4.41 
1 .86 
8.43 
3.18 
5.63 
2.74 
1 .59 



The PIAT was first given in spring '1978. 



^Tbe Child Rating Scale is oniy administered in the spring, thus there are no fall, 1976 figures 



Figure 1 a 



Plot of PDC and ComDarison GrouD Means on Measures in the 
Fall 1975, Spring 1977, and Spring 1978 Batteries 
(English-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1975 
scores on the same measure (see' text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 



Figure 1 b 
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Plot of PDC and Comoarison Group Means on Measures ir the 
Fall 1976, Spring 1S77, and Spring 1973 Batteries 
(English-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation or 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 
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Figure Ic 



Plot of PDC and Comoarison Srouo Means on Measures in the 
Fall 1976, Spring 1977, and Spring 1978 Batteries 
(English-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized vith reference to fall 1976 
scores on the same measure (see text for techmzii detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 
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Figure Id 

Plot of PDC and Comnarison GrouD Means on Measures in the 
Fall 1976, Spring 1977, and Spring 1978 Batteries 
(English-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation or 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 
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Figure le 



Plot of PDC and ComDarison Grouo Means on Measure in\he 
Fall 1976, Spring .1977, and Spring 197afBatteries 
(English-Dominant Analytic Samoije^ 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 
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Figure If 

Plot of PDC and Comoarison Group Means on Measures in the 
Fall 1976, Spring 1977, and Spring 1978 Batteries 
(English-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation o, 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 



Figure Ig 

Plot of PDC and ComDarison Grouo Means on Measures in the 
Fall 1976, Spring 1977, and Soring 1978 Batteries 
(English-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. l. 



Figure 2 a 



Plol of PDC and ComDarison Grouo Means on Measures in the 
f'all 1976, Spring 1977, and Spring 1978 Batteries 
(Spanish-Dorninant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 



Figure 2 b 



Plot of PDC and Comoarison Grouo Means on Measures in the 
Fall 1976 » Spring 1977, and Spring 1978 Batteries 
(Spanish-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. . 
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Figure 2 c 

Plot of PDC and ComDarison Group Means on Measures in the 
Fall 1976, Spring 1977, and Soring 1978 Batteries 
(Spani^h-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 
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Figure 2 d 



Plot of PDC and Coaiparlson Group Means on Measures in the 
Fall^ 1976, Spring 1977, and Soring 1973 Batteries 
(Spanish-Dominant Analytic Sample) 
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scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data for all timepoints shown. 
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Figure 2 e 

f 

Plot of PDC and Comparison Grouo Means on Measures in the. 
Fall 1976, Spring 1977, and Soring 1978 Batteries ^ 
(Spanish-Dominant Analytic Sample) 
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Note; Each of the means plotted is standardized with reference to fall 1976_ 
scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on^ases with complete 
data for all timepoints shown. 
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Figure 2\f 

\ 

Plot of PDC and Comparison Group iMeans on Measures in the 
Fall 1976, Spring 1S77, and Soring 1978 Batteries 
(Spanish-Dominant Analytic Sample) 
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scores on the same measure (see text for technical detail and explanation of 
exceptions). The means for each variable are based on cases with complete 
data, for all timepoints shown. 



Figure 2 g 



Plot of PDC and Comparison Grouo Means on Measures in the 
Fall 1976, Spring 1977, and Soring T978 Batteries 
(Spanish-Dominant Analytic Sample) 
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Note: Each of the means plotted is standardized with reference to fall 1976 
scores on the' same measure (see text for technical detail and explanation 
exceptions). The means for each variable are based'on cases with complete 
data for all timepoints shown. 
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year-to-year growth on the measure il'lustrated . And since 
all the means are standardized, growth* curves for one measure 
can be compared with growth curves for others (the "hill" 
is steeper for some). Note, however, that the group means 
plotted represent children's scores unadjusted for other 
factots (such as age, sex, or covariates related to test 
perfoirmance) . , In the analyses of program impact presented 
later in this report, adjustments have been made for a number 
of other factors extraneous to the effect of the PDC program. 

" Internal consistency . Table 11 presents internal consis- 
tency (Cronbach's alpha) coefficients for each measure in the 
spring 1978 battery. For the English-dominant analytic sample, 
all these , coefficients continue to exceed our criterion of 
.65, indicating that each one remains an adequately homogeneous 
measure of a central construct. In the Spanish-dominant sample 
coefficients have drop,p^d below the criterion level for BSM- 
Spanish, Verbal Fluency, and the "Dependence" scale of the CRS. 
Examination of the standard deviations of these measures 
(refer back to Table 10) reveals that they are lower in every 
case for the Spanish-dominant than for the English-dominant 
sample. And for the Spanish-dominant children, the standard 
deviations of BSM-Spanish and Verbal Fluency tend to be lower 
than they have been in past years. Thus the low reliability 
coefficients may reflect a condition of diminishing variation 
on these measures for Spanish-dominant children. And this, 
in turn, may be a consequence of diminishing sample size. 
The small kize of the Spanish-dominant • sample presents a 
general problem that will be discussed later in this report. 

Stability . Although we h^ve established no criterion that 
calls for a high correlation of scores from one testing period 
to another, these test-retest coefficients (Table 12) add to 
what we know about the properties \?f each measure, and thus 
constitute useful documentation. 

Relationship to "social competenge ." As in analyses of 
spring 1977 data, we examined the relaMonship of each measure 
in the spring 1978 battery to a set of n^asures established 
as proxy criteria for social competence. \(Only children from 
the English-dominant analytic sample entered, into these analyse 
due to sample size considerations.) The criteria consisted 
of the two subscales of the POCL ("Task Orienta^on" and 
"Soci oility") and the three subscales of the revised Child 
Rating Scale ( "Self-Assurance , " "Aggressiveness an4 "Depen- 
dence"). The rationale for selection of these criteria was 
that a broad range of the child's characteristics — social, 
emotional, cognitive, linguistic, and psychomotor — is spanned 
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Table 11 



Reliability of Child Measures : 
Cronbach's Alpha (Internal Consistency) 
PDC Spring 1978 Data 



Measures 


English-Dominant Children 


Spanish-Dominant Children 


COGWITIUE-LAWGUAGE 




a 






DOrl""Lliy 1 1 i>ni 


636 


Ji\ 


36 


. 8b 




14 


.95 


46 


.56 


Uov^Kal Pliionr'v/ 
VciUa 1 r 1 UcMLjr 


636 


.68 


47 


.51 


Verbal Memory-1 


636 


.73 


47 


.87 


Verbal Memory -3 


636 


.75 


46 


.79 


Draw-A-Child 


636 


.68 


47 


.75 


psvcHomroR 










Arm Coordination 


636- 


.72 


47 


.71 


SOCJAL-EUOnOMM 










POCL-1 : "Task Orientation" 


595 


.92 


42 


.97 


POCL-2: "Sociability" 


595 


.87 


42 


.93 


CRS-1: "Self-Assurance" 


620 


.90 


21 


.85 


CRS-2: "Aggressiveness" 


652 


.77 


22 


.84 


CRS-3: "Dependence" 


649 


.78 


21 


.55 



^Three instruments are not included: the Preschool Interpersonal Problem Solving Test and the Peabody 
Individual Achievement Test (Reading efnd Math subtests) do not lend themselves to computation of alpha, 
and the reliability of the Classroom Observation System was determined differently. 



Texas and California only (bil ingual/bicultural demonstration sites). 



53 



53 



Table 12 



Stability (Test-Retest Correlations) of the Child Measures at Three Timepoints: 

Fall 1976, Spring 1977 and Spring 1978 



Measures 



COGNITIVE- LANGUAGE 

BSM-English 
BSM-Spanish 
Verbal Fluency 
Verbal Memory-1 
Verbal Memory-3 
Draw-A-Child 

PSYCHOMOTOR 

Arm Coordination 

SOClAL-EMOTlOWAi. 



PIPS 

POCL-1: "Task Orientation" 
POCL-2: "Sociability" 
CRS-1 : "Self-Assurance"^ 
CRS-2: "Aggressiveness"^ 
CRS-3: "Dependence"^ 



English-Dominant Children 


Spanish-Dominant Children 




F76-S77 


S77-S78 F76-S78 




F76-S77 


S77-S78 


F76-S78 


N 


^t 


^t 


^t 


N 


^t 


^t 


•^tt 


595 


«75 


.fay 


.fa 1 


6 


.64 


.62 


.50 


5 


.88 


. yo 


OA 

. y't 


41 


.60 


.74 


.38 


586 


.48 


• oc 




45 


.62 


.23 


.11 


596 


.49 


. Hy 


. oo 


45 


.67 


.53 


.42 


588 


.49 


.33 


.30 


42 


.55 


.30 


.41 


587 


.56 




4? 


45 


.27 


.27 


.10 


585 


.42 


.40 


.34 


45 


.31 


.40 


.51 


587 


.41 


.32 


.20 


'45 


.11 


.23 


-.03 


528 


.42 


.36 


.19 


40 


.49 


.17 


.26 


528 


.44 


.39 


.25 


40 


.39 


.14 


.26 


464 




.40 






ineuffiaient aample 




495 




.32 






inauf fiaient econple 




497 




.08 






ineuffiaient ample 





^The Child Rating Scale was not administered in fall, thus a test-retest coefficient can be computed 
for spring time points only. 



when the teacher and tester assess the child's social behavior, 
and that these ratings thus reflect the integration of the 
elements of social competence — the configuration of traits 
that contribute to the child's everyday effectiveness. 

The procedure followed in these analyses was to regress 
each separate measure from the spring 1978 battery on a set 
of predictors consisting of the proxy social competence 
criteria (the scales of the POCL and CRS) plus age, sex, and 
ethnicity (included as covariates) . Interpretation of the 
results focused on the degree to which the "social competence" 
criteria contributed to prediction of test scores beyond 
the prediction obtained using the covariates alone. The greater 
the predictive value of the proxy criteria, the greater the 
apparent relevance of the dependent measure to the construct 
of social competence. 

All the measures examined bore a statistically significant 
relationship to the criteria, as can be seen in Table 13. 
Fairly strong relationships were found for Verbal Fluency, 
Verbal Memory- 3, PIAT-Math, and PIAT-Reading . Moderate rela- 
tionships were found for Verbal Memory-1 and Draw-a-Child . 
The criteria were rather weakly related to the remaining 
measures: BSM-English, Arm Coordination, and PIPS. 

Correlations among measures . For the sake of further 
documenting the psychometric properties of the measures, 
Table 14 presents a matrix of intercorrelations for the 
English-dominant sample. Table 15 presents a corresponding 
matrix for the Spanish-dominant sample. 

Factor structure. In Tables 16 and 17, the relationships 
depicted in the correlation matrices are reduced to a smaller 
number of common factors found to underlie the English and 
Spanish batteries. For the English-dominant sample-,- five 
factors emerged. The first, which might be labeled "general 
cognitive," consists primarily of BSM-English, Draw-a-Child , 
PIAT-Math, and PIAT-Reading. The second factor consists 
mainly of two of the CRS scales-^ "Self-Assurance" and "Depen- 
dence." Since the second measure is negatively loaded, the 
factor may essentially represent a broader measure of self- 
assurance than is contained in CRS-1 alone. Factor 3 is based 
mainl' on Verbal Fluency, Verbal Memory-1, Verbal Memory- 3, 
and PxPS. These all share an emphasis of linguistic fluency, 
and, indeed, "linguistic fluency" might be an appropriate 
name for the third factor. Factor 4 consists of tha two sub- 
scales of the POCL, and perhaps represents the "halo' common 
to these two measures, which are aggregates of ratings assigned 
by the same tester. Arm Coordination and the "Aggressiveness 



4S 



ibl 



OS 



Relationship of "Social Competence" Criteria and Background Variables to Test Scores of lOHerg^r 



fiShlpfS Tes?Scor^Pof BSergSrRn (^ffdref 





% of variance accounted for 






% of variance accounted 
for jointly by "social 
competence" criteria & 
backaround variables 


•by "social competence" cri- 


Significant predictors 
(p<.05) & their partial 
correlations with test scores 


Test 
(N = 479) 


teria beyond background 
variables 


BSM-English 


.'24** 


.02* 


Variable Partial 
Ethnicity -.41 
POCL-2 .10 
Age .10 


Verbal Fluency 


. Id 


.15** 


POC -1 .19 
POCL-2 .13 . 
CRS-1 .12 


Verbal Memory -1 


# 

.08** 


07** 


POCL-1 . vl8 


Verbal Memory-3 


.15** 


.11** 


POCL-1 .22 
Age .15 


Arm Coordination 


.17** 


.02* 


Sex -.29 
Aqe .18 
Ethnicity .18 


Oraw-A-Child 


.11** 


.08** 


CRS-1 .21 
Sex .10 
Age .09 


PIPS 


.04* 


.04* 


CRS-1 .10 


PIAT-Math 


.25** 


. 1 0 


Ethnicity -.28 
POCL-1 .24 
CRS-1 .19 
Age .16 


PIAT-Reading 


.21** 


.15** 


POCL-1 .27 
CRS-1 .20 
Ethnicity -.20 
Age .10 



♦probability of associated F ratio ^ .05 
Q ^*probability of associated F ratio _ .0001 

ERIC 
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KEY: 

POCL-1: "Task Orientation" 
POCL-2: "Sociability" 
CRS-1 
CRS-2 
CRS-3 



" Self-assurance" 
" Aggressiveness" 
"Dependence" 



G 



Table U- 

IntercorreTatlorfe- of Child Measures foV^ English-Dominant Children^ 

PDC Spring 1978 Data » 





-Engl ish 


-Spanish 


bal Fluency 


•bal Memory- 1 


CO 

1 

i 

s: 


■o 

!£ 
o 
1 

i 

2 


JCZ 
•M 

ra 

SI 
1 

h- 


-o 
ra 

0) 

fV 

i 


Drdination 


t/1 
a. 


1 

.-J 
o 


CL-2 


T 

o 


1 

a 

1 

O 


ro 
t 




CHILD MEASURES 


Ul 
CQ 


tn 

CO 






0) 


o 


«aC 
O- 




t O 
< o 


O- 


o 

CX- 


o 

O- 


o 




BSM-English 




-.03 

10 


.25 

636 


. 22 

636 


.18 


. 29 


.32 

5i5 


. 33 

600 


— . U J 

535 


. Do 

535 


. 1 J 

bUb 


.21 

bOb 


553 


nn 
by4 


.01 

5/2 




BSM-Spanish 


~. 03 

19 




. £ J 

19 


. CO 

10 


. 23 


. U4 

19 


.49 

18 


O T 

16 


.20 

19 


. U/ 


. OD 

7 5 


.26 

i5 


11 

lb 


- . 18 

iff 


-.06 

i5 


.ANGUAGE 


Verbal Fluency ^ 


.25 

636 


.23 

29 




.39 


.39 


.26 


. 32 

61b 


. J J 

6tW 


.14 

535 


97 

63(}_ 


. J J 

bOb 


.26 

.^^^5 - 


. CO 

bb3 


.02 

b74 


- .16 


Verbal Memory- 1 


.22 

626 


.28 

19 


.39 




.34 

636 


. 1 7 

535 


. 21 

61b 


600 


.01 

636 


91 

5.^5 


bOh 


.09 

b9b_ 


1 Q 

bb3 


5X'^ 


- . 10 

b7P. 


1 

UJ 

> 


Verbal Meinory-3 


.18 

630 


.23 

10 


.39 


.34 

636 




.20 

636 


5i£ 


. 1 / 

000 


.11 

630 


1 O 
c'35 


. J J 


.21 

19 b 


. 1 9 

_553_ 


.01 

b/'i 


-.14 

b/2 


C0GNI1 


Ordw-A-Child 


.29 

636 


.04 

19 


.26 

636 


.17 


.20 




.30 

6ib 


.31 

600 


.04 

636 


.11 

630 


.14 


.09 


.28 

bi.3 


-.02 


-.15 

_57^ 


PIAT-Math 


.32 

61b 


.49 

18 


.32 

di5 


.21 


.29 
5i.') 


,30 

6li) 




.53 ^ 


.09 

'"^ti'ib" 


.10 

61b 


.34 

t>/b 


.24 


.28 

535 


-.02 

'ob6 _ 


-.09 

biiS 




PIAT-Reading 


.33 

600 


.23 

16 


.33 

600 


.23 

600 


.17 

600 


.31 

600 


.53 

56>t? 




.01 

600 


.12 

600 


.35 

bOl 


.20 

161 


.30 


-.03 

b'1l 


-.11 

t^'lO 
















1^ 





• , ,, . 


PSYCHO 
MOTOR 


Arm Coordination 


-.03 

630 


.20 

19 


.14 

636 


.01 

636 


.11 

636 


.04 

535 


.09 

5/5 


.01 

600 




.03 

636 


.09 

bob 


.01 

bob 


.11 

553 


,08 


-.11 

5/2 






























PIPS 


.08 

6.^6 


.07 

lU 


.27 


.21 

636 


.18 

636 


.11 

636 


.10 

61b 


.12 

600 


.03 

636 




. 1 3 

5i//) 


.13 

bOb 


.15 " 

553 


03 
5/'l 


-.11 

5/2 




POCL-1 : "Task Orientation" 


.13 

LJb 


.56 

15 


.33 

bOb 


.23 

bOb 


.33 

b9b 


.14 

bOb 


.34 

trr'b 


.35 


.09 

bUb 


bUb 




.51 

bOb 


. c 1 

bib 


535 


-,16 

535 


lOIlOV^ 


POCL-2: "Sociability" 


.21 

bus 


.26 

lb 


.26 

bUb 


.09 


.21 


.09 

bUb 


bib 


.20 

;)5i 


.01 


.13 

bUb 


.51 

bOb 




.06 

bib 


.20 

535 


.00 

535 


lAL-EI 


CRS-1: "Self -Assurance" 


.09 

bbS 


.11 

lb 


.25 


.19 

bb3 


.19 

bb3 


.28 

i'ii')3 


.28 

_b3b^^ 


.30 

bill 


.11 

bb3 


:15 

bb3_ 


.27 

_M/^ 


.06 




-.15 

55^7 


-.42 

b-IH 


DOS 


CRS-2: "Atjgressi veness" 


.00 


-.18 

16 


.02 


-.07 




.01 

_b/i_ 


-.02 

b?i__ 


-.02 


-.03 




.03 


-.04 

b36^ 


.20 

^^■5 


-.15 

J^oO _ 




.07 

b69 




CRS-J: "Uependonce" 


.01 

I } 1, 


-.06 

lb 


-..16 


-.10 


-.14 


-.15 


-.09 


-.11 

oiO 


-.11 

6 / 


-.11 

b/ii_ _ 


-.16 


.Of) 

I 3i _ 


-.42 

_ M/i 


.07 





ERIC 



^Sample size for each correlation is shown in italics. 
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Table 15 

Intercorrelatlons of Child Measures for Spanish-Dominant Children^ 

. PDC Spring 1978 Data 



CO 







BSM-Spanish 


jr. 

10 
•r- 

'o) 
c 

LU 

LO 
CO 


Verbal Fluency 


Verbal Memory-1 


, Verbal Memory-3 


'O 

•r- 
_CZ 

o 

«t 
i 

(O 

Q 


4-» 

s: 

a. 


PIAT-Reading 


Arm 

Coordination 


PIPS 


i 

—1 ■ 
o 

a. 


POCL-2 


r— 

O 


CRS-2 


CRS-3 


1 




CHILD MEASURES 


COGNITIVE-LANGUAGE j 


BSM-Spanish 




-.03 

36 


.48 

46 


.43 

46' 


.32 

45 


• JD 

46 


26 


.37 

26 


-.11 

46 


.38 

*i 0 


.31 

41 


.19 

4 1 


.23 

40 


.15 

45 


-.46 

44 


BSM-Engllsh 


- 03 _ 

35 




-.03 

66 


.47 

66 ' 


.22 

66 


- . Do 

66 


9/1 

26 


.30 

26 


-.17 

66 


.11 


.28 

61 


.34 

61 


.02 

3J 


.57 

35 


.08 
64 


Verbal Fluency 


.48 

46 


-.03 

26 , 




ir 

.28 

4? 


.21 


.33 

47 


. 4 b 


.39 

26 


-.16 

47 


.03 

y ( 


. 29 

42 


.30 

42 


.05 

41 


.00 

46 


-.02 

45 


Verbal Memory-1 


.43 

46 


Al 

66 


.28 

■ 47 




.42 

46 


n o 

47 


. J 1 

l:6 


.24 

26 


-.10 

4'7 


.22 


.53 

42 


.39 

42 


.00 

4i 


.43 

46 


-.10 

45 


Verbal Memory-3 


.32 

45 


.22 

16 


.21 


.42. 

46 




. U 1 

46 


. 4 I 


.33 

26 


.18 

46 


-.44 

•ir- 


.33 

41 


.32 

41 


-.08 

40 


.32 

45 


-.05 

44 


Draw-A-Child 


.36 

46 


-.06 

'66 


.33 


.20 

47 


.01 

40 




.22 

26 


.09 

26 


.08 

47 


.14 

47 


.12 

42 


.04 

_42 


.25 

41 


.00 

46 


-.04 - 

45 


PIAT-Math*^ 


.42 


.24 


.45 

26 


.31 

26 . 


.41 

26 


.22 




.44 

26 


.09 

26 


.26 


.29 

21 


.32 

21 


.56 

*> 0 


.51 

26 


.05 

25 


PIAT-Reading*^ 


.37 

26 


.30 

26 


.39 


.24 

26 


.33 

2b' 


.09 

26 


.44 

26 




.17 

26 


.19 

26 


.10 

21 


.11 

21 


.36 


.19 

26 


.00 

25 




■ 




















PSYCHO 
MOTOR 


Arm Coordination 


-.11 

46 


-.17 


-.16 

47 


-.10 

4? 


Vl8 
46 


.08 

47 


.09 

^6' 


.17 

26 




.06- 

47 


-.40 

42 


-.46 

42 


-.08 

41 


-.05 

46 


.04 

45 






















«t 

o 

liJ 
1 

<c 

o 
o 

LO 


PIPS 


.38 

46 


.11 

S6 


.03 

47 


.22 

47 


.44 

46 


.14 

47 . 


.26 

26 


.19 

26 


.06 

47 




.36 

42 


.29 

42 ■ 


.17 

41 


.06 

46 


-.27 

45 


POCL-l : "Task Orientation" 


.31 

41 


.28 

61 


.29 

' 42 


.53 

42 


.;33 

41 


.12 

42 


.29 

21 


.10 

21 


-.40 

42 


.36 

42 




.85 

42 


.30 

3/ 


41 


-.38 

41 




POCL-2: "Sociability" 


' .19 

41 


.34 

61 


.30 

42 


.39 


.32* 

41 


.04 


.32 


.11. 

21 


-.46 

42 


.29 

42 


.85 

42 




.29 


.29 

41 


-.20 

41 


CRS-1 : "Self-Assurance" 


.23 

40 


.02 

61 


.05 

4i 


.00 


-.08 

4J 


.25 

_ '11 


.56 

''2 

.51 

_ 20 

.Ob' 


.36 


-.08 

41 


.17 

41 


.30 


.29 

67 




.18 

41 


-.08 

_40 


CRS-2: "Aggressiveness" 


.IS 

45 


.57 

3/i 


.00 


.43 


.32 

45 


.00 

^jr6 

-.04 

45 


.19 

26 


-.OS 

46 

.04 

45 


.06 

46 

-.2/ 


.28 

41 


.29 

41 


.18 

41 




-.13 

45 




CRS-3: "Dependence" 


4 1 


-.08 

64 


-.02 

45 


-.10 

45 


-.OS 

44 


.00 

w 5 


-.38 

41 


-.20 
'U 


-.08" 
40 


-.13 

45 






0 
1 


^Sample size for each cofrel 
'^Administered In English.* 


ition is shown in italics. 
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Table 16 

Factor Analysis^ of Scores on Child Measures, English-Dominant Children 

Spring 1978 



- 


Fcictor Loading of Child Measures 


N = 479 




(higheat loading italioized)- 




\Child Measure 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


BSM-English 


* Oo 


-.20 


.21 


.03 


- 0? 


Vfi^^ba] Fluency 




,15 


.58 


.25 


.04 


Venbal Memory-1 


21 


-.01 


. 73 


.01 


-.19 


Verbal Memory-3 


1 R 


.14 


.62 


.20 


.06 


Arm Coordination 




.43 


.10 


.03 


.52 


Oraw-A-Child 


, 00 


.22 


.12 


-.15 


.19 


PIPS 




.05 


.62 


.03 


.11 


PIAT-Math 


. 65 


.15 


.11 


.33 


^,05 


PIAT-Reading 


. 72 


.12 


.03 


.30 


-.12 


POCL-1 : "Task Orientation" 


.16 . 


9 1 

,c 1 


91 


m OU 


-.11 


POCL-2: "Sociability" 


.12 


r.l3 


.13 


.80 


.22 * 


CRS-1: Self-Assurance 


.26 


. 7Z 


.11 


.11 


-.15 


CRS-2: "Aggressiveness" 


.06 


-.24 


-.02 


.07 


.81 


CRS-3: "Dependence** 


.00 


-. 77 


-.11 


.01 


.00 


% of Total Variance 








11.5 


8.0 


Accounted for 


15.2 


11.5 


13.1 



^Principal components solution, varimax rotation 
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Table 1-7 ' 

Factor/ Analysis^ of Scores on Child Measures, Spanish-Dominant Children 

Spring 1978 



N =^ SS 


Factor Loading of Child Measures 

^^(hiqhest loadinq italicized) 




Child Measures 


ractor 1. 


^ "f* ✓ 

ractur c 


Pa r f" n r '\ 


Factor 4 




BSM-Spanish 


.34 


.08 


.63 : 


-.12 


.'46 


Verbal Fluency 


,'.45 


.36 




o n 


-.11 


Verbal Memory-1 


. 69 


. 03 




. HO 


.00 


Verbal Memory-3 




.06 


-.03 


.08 


.31 


Arm Coordination 




-.82 


.07 


.13 


.07 


Draw-A-Child 


.01 


-.10 


.82 


.08 


.09 


PIPS ' 


.35 


-.02 


.23 


.06 


. 6? 


POCL-1 : "Task Orientation" 


;36 


.69 


.14 


. 34 


.35 


POCL-2: "Sociability" 


■ .30 


.80 


.08 


.30 


.17 


CRS-1 ; "Self -Assurance" | 


-.38 


.21 


.56 


.48 


.14 


CRS-2: "Aggressiveness" | 


.20 


.11 


-.05 


.81 


.03 


CRS-3: "Dependence" 


.03 


-.14 


-.02 


-.04 


-.87 


% of Total Variance 1 
Accounted for 1 


17.2 


16.9 


15.9 


11.9 


14.2 



^Principal components solution, varimax rotation. 
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scalX of the CRS are the principal constituents of factor 5. 
The relationship between these two measures is not sufficiently 
obvious to warrant naming of the factor, but one plausible 
connection between the measures is an emphasis on physical 
maturity; the children who score highest on psychomotor t^sks 
such as Arm Coordination are likely to be the same children 
who are physically ascendant, and thus "aggressive," in school. 

As in the past, th^ factor structure found for the Spanish- 
dominant sample (Table 17) is similar to that for the English- 
dominant sample, yet not identical. However, the size of the 
sample available for this analysis (35 children) has reached 
the point at which factors are likely to be highly unstable 
from one replication to another. Thus interpretation of these 
results in isolation from other findings is inadvisable. 



Characteristics of the Classroom Observation System 

Background . At the outset of the PDC evaluation, High/Scope 
undertook to develop a time-sampling classroom observation system 
that focused on the behavior of individual children. The system 
that eventually emerged involved observing each child |s class- 
■room activities for two five-minute periods and tallying 
certain focal behaviors. The observation categories can be 
described broadly as: nature of classroom involvement, nature 
of verbal behavior, nature of interactions with peers, and 
natu:ce of interactions with adults. 

Adaptation of the Observation System as a measure of 
classroom atmosphere . Past analyses have revealed that the 
measures produced by the Observation System have only a tenuous 
relationship with other child measures, even when observation 
variables were correlated with teachers ' ratings of similar 
'dimensions. The likely explanation is that the brief sampling 
of classroom interactions does not adequately reflect these 
dimensions of a child's behavior. Thus it was decided that, 
in the future, observation data would be regarded as reflec- 
tions of classroom atmosphere rather than as reflections of 
the .raits of the individuals observed. In essence, what we 
obse ^ed among children in the classroom was to be taken as ^ 
an aspect of the classroom's personality rather than the child s. 
(There is precedent for this approach in the literature on 
unit-of-analysis decisions in educational research.) 
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We arrived at classroom-level measures by selecting PDC 
and comparison classrooms in which at least three children had 
been observed in spring 1978 and, within each of those class- 
rooms, averaging the values of the observation variables across 
children. The final sample available for analysis consisted 
of 39 PDC and 42 comparison classrooms. 

Results of preliminary analyses . Following the pattern 
established for examination of other PDC measures, our first 
analyses of the Observation System focused on its psychometric 
properties. The main purpose of these analyses was to examine 
the relationship of observation variables to each other and 
to other measures in the PDC battery, also averaged across the 
children within each classroom. The resulting correlations 
were considerably lower than those that have been found for 
other PDC measures — so low that they cast doubt on utility of 
the observation variables as predictive or explanatory variables. 
Nevertheless, we proceeded to an analysis of the differences, 
between PDC and comparison classrooms on these variables, more 
for psychometric purposes than for purposes of studying program 
impact. That is, since PDC and comparison classrooms are expected 
to differ in atmosphere, a finding of significant differences 
on the observation variables would suggest that the system had 
succeeded in portraying some important dimensions of that atmos- 
phere. A finding of no difference on the observation variables 
would leave open the question of whether the null finding was 
due to invalidity of the measures or to a genuine absence of 
differences. 

We performed two series of analyses of variance in examining 
the differences between PDC and comparison classrooms on the 
seven main variables produced by the observation system. In 
the first series, the dependent measures were analyzed in the 
form of raw percentages — each variable expressing the proportion 
of all observed occasions on which a critical event occurred 
(e.g., percent of the total time that children were involved 
in social behavior) . But since percentage data tend not to 
conform to the homogeneity-of-variance assumption fundamental 
to analysis of variance, we created a second set of variables 
equal to : 

1/2 

(2 (ai^csin (original variable) ) ) . 

This ,,rans format Lon, suggested by Winer, "is effective in 
stabilizing the variances" ^ of the original variable across 
desiiin cells.* 




finer { J. Star istical principles in experimental design . 
New York: McGraw-Hill, 1971. Pp. 399-400. 
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Subsequent analyses of both the transformed and untrans- 
formed sets of variables revealed no differences whatever 
between PDC and comparison classrooms. Thus r since it cannot 
be determined whether this null finding is due to the actual 
absence of classroom differences or simply to the. in-ioility 
of the observation system to detect such differences, the 
analyses will not be further documented. 

From spifl'ng 1979 onward, a new observation system will 
be used in the PDC evaluation. This system, to be described 
in future reports, is expected to provide greater insight into 
classroom processes and greater measurement reliability than 
the system employed through spring of 1978. 



IV 




EXAMINATION OF TREATMENT EFFECTS 



Guiding Questions 

There are many facets to the question, "Has PDC had an 
impact on children?" Each facet gives voice to a different 
emphasis : 

• Has PDC had the same impact on okildven at all sites? 
m Is FDC's impact evidenced equally on all measures? 

• Is FVC's impact the same for all children^ regardless 
of background? 

• Eas PDC had an impact since the beginning of the Read 
Start year? Since the beginning of the kindergarven 
year? 

Clearly there can be no single answer to the question of 
PDC's impact, since it is not really a single question. Thus 
we took several different approaches to analysis of spring 
1978 data, each approach covering a somewhat different aspect 
of the central issue. 

Analytic Design 

The basic design of the impact analyses reported here was 
similar to the design used for analysis of impact through the 
Head Start year: by means of analysis of covariance, spring 
1978 test scores were each adjusted for prior scores on the 
same measures (except in the repeated measures analyses) , 
for fall^ 1976 WPPSI^ score, and for the factors of site, 
ethnicity, sex, and prior preschool experience. Each analysis 



^The Block Design subtest of the Wechsler Preschool and Primary 
Scale of Intelligence (WPPSI) was administered to all children 
at the tim^ of program entry (fall 1976) as a baseline measure 
of general cognitive aptitude. This fall 1976 measure con- 
tinues to ' serve in our analyses as a covariate representing 
one dimension of children's initial ability. 
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then yielded an assessment of the difference between PDC and 
comparison groups on the adjusted score. This design allowed 
evaluation of both the main effect of the PDC program and the 
interaction of the program effect with site conditions. That 
is, we exam.ined first the question of whether PDC had, on the _ 
whole, produced an effect for PDC children relative to comparison 
children, then the question of whether this effect was the 
same at all sites. 

In concise terms, the analytic model represents a propo- 
sition that. . . 



An individual child's status on a 
given measure in spring 1978 



= The sum of the contributions of: 
Grand mean 

Mean spring 1978 status of all 
children on the measure under analysis. 

+ Covariates 



Baseline performance status (fall 1975 
WPPSI score and, in some analyses, fall 
1976/spring 1977 status on the measure 
xinder analysis) . 

+ Program^ factor 

The child's educational program (PDC 
or comparison) . 

4- Site factor 

Conditions associated with individual 

sites . 

4- Sex factor 

Conditions associated with being a boy 

or a girl. 

4- Preschool factor 

Preschool experience prior to the 
1976-1977 Head Start year. 

4- Program-by-site interaction term 

Conditions associated with local programs 
as they differed from national programs. 

4 Error term 

Variation within the categories defined 
by this design, due to factors not 
specified, plus variation due to un- 
specified interactions of the design 
factors * 
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For each of the two analytic samples of children (English- 
dominant and Spanish-dominant) , three separate analyses were 
carried out for each measure in the spring 1978 battery^ — a 
total of 68 analyses. The three series of analyses all fit the 
framework just described but addressed three somewhat different 
questions: 

1. Has a group difference arisen since fall of the Read ^ 
Start year? 

(Approach: .analysis of covariance with two covariates 
and one dependent variable.)^ 

2. Has a group difference arisen since spring of the Head 
Start year? 

(Approach: analysis of covariance with three covariates 
and one dependent variable . ) 

3. Is there a difference in the growth curves of the two 
groups? 

(Approach: repeated measures analysis of covariance 
with one covariate and three simultaneous dependent 
' ' variables . ) ^ 

The first question is probably the primary one for the PDC 
evaluation. For the respective kindergarten-year measures 
analyzed, each child's fall 1976 score on the same measure served 
as a covariate, along with the child ;s fall 1976 WPPSI score. 
The analysis then focused upon change in the dependent measure 
since fall 1976, allowing estimation of PDC children's gains 
relative to comparison children during the Head Start and kinder- 
garten ]<(fcrs. 

« 

The second question is logically subsidiary to the first; 
It basically asks whether PDC-versus-comparison differences 
have become manifest during the kindergarten year. If they have, 



^Due to the small size of the Spanish-dominant analytic sample 
(N=46) it was necessary to reduce the number of parameters 
involved in the analyses. Thus the design for this sample 
excluded the factors of site, ethnicity, .sex, and prior preschool 
experience. 

^Because the initial match between PDC and comparison groups was 
quite close, as noted in this and past reports, the purpose of 
using covariates ' to "adjust" scores is not so much to equalize 
the initial status of the PDC and comparison groups as to lend 
greater precision to the ^ipalyses, by reducing the amount of 
variation due to unspecified "error." ^ 
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then, theoretically, the differences should be reflected in 
analyses addressing the first question as well, since those 
analyses tell whether differences have arisen at any time over 
the course cf the program. However, since the second series 
of analyses introduced a new covariate (spring 1977 score on 
the dependent measure) not present in the first series, the 
statistical result could conceivably indicate a group difference 
for the kindergarten year when none was indicated for the Head 
Start and kindergarten years together. 

The third question, concerning growth curves, was addressed 
by performing repeated measures analyses of covariance. The 
results of these analyses should be positive if there are 
differences between PDC and comparison children (after adjust- 
ment for extraneous factors) at any point from fall 1976 to 
spring 1978. Logically, one would expect these analyses to 
indicate a PDC-comparison difference for any measure that 
reflected a group difference in the first .or second series of 
analyses. However, because we interpreted these analyses 
conservatively^, it is possible for the results to appear 
somewhat inconsistent with the results of the preceding analyses. 



^One of the assumptions involved in univariate repeated measures 
analysis is that all possible pairs of the repeated measures 
are equally correlated. (This is known as the assumption of 
"compound symmetry" of correlations.) In PDC' s case, as in the 
case of most extended longitudinal studies, this assumption 
often is not sustained by the data. Thus, in examining the 
results of the repeated measures analyses, we chose not to 
interpret the evidence bearing on the equivalence or non- 
equivalence of the slopes of PDC and comparison group growth 
curves. Instead, we based our conclusions about program effects 
on that part of each repeated measures analysis that tested 
the difference between the PDC and comparison means for all 
repetitions of the measure analyzed (e.g.: (fall 1976+spring 
1977+spring 1978 score)/3)). This meant that, in the interest 
of statistical prudence, we examined only one of the two wa 
in which the growth curves of the groups might differ. In 
the future, we will have access to computer programs that will 
circumvent the compound symmetry assumption by performing 
repeated measures analyses in a multivariate mode. This will 
allow us to deal with the question that we had to bypass 
here: the question of a possible difference in the slopes 
of the PDC and comparison curves. 




"-^ Results of the Impact Analyses ^ ' ^ ^ 

The English'dorainant analytic sair^ple . The results of the 
three series of analyses for the Engiish-doir.inant sample are 
summarized in Table 18. The prevailing pattern evident there 
is one of no significant differences between PDC and comparison 
groups: the first series of analyses revealed no overall 
differences ("main effects," in ANOVA terms) on any measure, 
and the second and third series each indicated a marginally 
significant overall difference on just a single measure. 
There are more numerous indications of group differences local 
to particular sites ( "program-by-site interactions") , but 
the three series of analyses are not in close agreement on 
these. On only one measure out of the 12 commonly examined 
do the three analyses consistentl,y reflect a significant 
difference. That measure is the "Aggressiveness" subscale*of 
the Child Rating Scale. Analyses 1 and 2 indicate that there 
are PDC-comparison differences in "Aggressiveness" at some 
but not all sites, while analysis 3 indicates a significant 
overall group difference and no site-level departures from 
that effect. 

Figures 3a through 3g portray the relative status of PDC 
and comparison groups on every measure re/eiving a "Yes" any- 
where in Table 18. Each of these figures shows, first, the 
status of all PDC and all comparison children combined. Then, 
where there was any indication of a group-by-site interaction, 
each figure shows plots of group means for the four sites 
where groups were fouiid to depart most substantially from the 
overall' pattern. Some insight into the nature of an interaction 
can be obtained by comparing the plot for each site with the 
plot for all sites combined. For example, in Figure 3a it can 
be seen that, overall, PDC children's scores on Verbal Memcpry-1 
are lower than those of comparison children at all three time- 
points. In Utah and West Virginia, however, PDC children tend 



^Because of limitations inherent in the computer program used 
for these analyses (prog3?km P2V of UCLA's BMD series) , it was 
necessary to Exclude Georgia children from all the analyses 
reported here- This was due to the program's inability to 
tolerate the absence of a comparison group in Georgia. Our 
imminent acquisition of a more tolerant, more general , program 
will eliminate this problem in the future, allowing inclusion 
of Georgia data. 

^Appendix B dociaments the significance levels associated with 
the contributions of each covariate and design factor in|jeach 
of the major analyses described here. 
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Table 18 

Summary of Analyses of Program Effects for the English-Dominant Analytic Sample 
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Analytic Question: 


Analysis #1 : 

"Has a group dij'fei'enoe 
aviaen ainae fall of the 
Head Start year?" j,^ 


Analysis #2; 

"Has a group differenae 
ariaen aince spin-ng of 
the Head Start year?" 


Analysis #3: 

"i8 there a difference 
in the growth curn)e8 
of the two groups?" 




Treatment group, site, 
ethnicity, sex, prior 
preschool experience 


Treatment group, site, 
ethnicity, sex, prior 
preschool experience 


Treatn^ent group, site, 
ethnicity, sex, prior 
preschool experience 


Covariate(s) : 


F76 WPPSI, F76 status 
on the target measur^ 


F76 WPPSI, F76 status 
on the target measure^, 
S77 status on the 
target measure 


F76 WPPSI 


Dependent Variable(s): 


S78 status on the 
target measure 


S78 status on the 
target measure 


F76, S77, and S78 
status on the 
. target measure 


Target Measui^e 


Sample 
Sized 


All Sites Individual 
Combined Sites 


All Sites Individual 
Combined Sites 


All Sitei Individual 
Combiner Sites 


BSM-English 
Verbal Fluency 
Verbal Memory-1 
Verbal Memory -3 
Arm Coordination 
Draw-a-Child 
PIPS 

POCL-1 :"Task Orientation" 

POCL-2: "Sociability" 

PIAT-Matha 

PIAT-Readinga 

CRS-1 :"Self-Assurance"D 

CRS-2: "Aggress iveness"b 

CRS-3:"Dependence"'^ 


568 
559 
569 
561 
558 
560 
560 
504 
504 
586 
572 
420 
449 
451 


No No 
No No 
No No 
No Yes** 
No No 
No No 
No No 
No Yes^ 
No No 
No Yes** 
No Yes** 
No No 
No Yes** 
No No 


No No 
No No 

No No 
No Yes** 
No No 
No No 
No No 

INO HO 

No No 

Yes* No 
No Yes* 
No No 


No No 
No No 
No Yes* 
No No 
No No 
No No 
No No 

Nn Nn 

No No 

No No 
Yes* No 
No No 



Administered in S78 only 
^Number of PDC and comparison children with complete data on all variables involved. 



*p 
**p 



< ,05 (two-tailed) 

< ,01 (two-tailed) 



jt>0 



y 
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Itlustration of Mam P>'ogr3m Effects and Program-Cr/'Site Interactions for Verba! Memory - 7 

(Engiish-Dominam Analyvc Satnple} 
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Figuris 3b 

liiustrmon of Ma/ri Program Effects and Pragram-by-Site interactions for Verbal Memor/ - 3 

(Engii^-Dommant Analytic Sample) 
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The poifTQ graphed nprewnt group means adjujted for fall 1976 WPPSI seen 'and for the effects of ^ex, etnnjcitv. 
«xl prior preschool expenence/The meam are standaraized on the basis of the overall mean and standara ilavutjon 
for th« e jriiest testing period shown. 
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Figure 3c 



lllustrBtion*of Main ProgrBm Effects and P^ogram-by-She Inxemcxions forPOCL-l: 
Task OrientBVon" (Engiish-Dominani Anaiyvc Sampie) 
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Figure 3d 

llluszratjon of Main Program Effects and Program- by -St re Interactiorts for PIAT-Math 
(English-Dominam Anaiyvc Sampie j 
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Mcmu The poinu graphed represent group means adiarted for fall 1976 WPPSI score and for the effecu of x, etnn cirv, 
«id prior prescfiool experienca. The means are standaroued on- the basis of the overall mean aiid stanaarc djvation 
tor the earliest tasting perioo shown. ^ * 
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Figure 3e 



iflustm'on of Main Pr vyrd fri Effects and Program-by -Site fmerxruons for PI A T-Readirig 
(Engihh'Domirmit Anaiyric Sampl&l V_ 
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Figure 3f 

Itiusamon of Main Program Effects for C/?S- 7; '^If- Assurance'' 
(Engiish'Dominant AnaJytic Sampie} 



n 

e 
% 



2 0 



I 



--mPDC 

"^Comparison 



Ail Sitei Combtned 



No group-by-sita intBractiom indicated 



i_L 



S77 S78 



Eric 



111* pf?«**^ gnishad mp^MHt ^raop 
awd prior prtacnooi «xp»n«nca. !>• 
r wvttng Period slujwru 



T«t»oPtnQd 

adkrad far fall t«76 ¥iPPSI orw and foritia ^effects of «. etnrridtv, 
ar« mndarctzad oo bncs of tha ovanil rrMn and sianaarc oeviation 



62 



83 



Figure 3g 



lUuszrwdon of Main ProgrBm Effecxs and Rrogram-by-Site Interactions fvrCRS-2: 
"^Aggressjveness"" (Engiish-Dominant Ans^yvc Saniptej 
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to be doing relatively better, while in Michigan and Washington 
they '.re doing relatively wor§e. It's interesting to note that 
Utah and West Virginia show up positively in three other figures: 
3b' (Verbal Meniory-3) , 3c (POCL-1) , and 3d (PIAT-Math) . 

The edge that the PDC groups in Utah and West Virginia 
appear to have over PDC groups in other sites stands out 
against a background of findings that otherwise tend to be 
either null or inconsistent. As far as the English-dominant 
sample goes, only a few other findings are worthy of note. 
First, in no site is the FDC group's adjusted score on PIAT- 
Reading higher than the comparison group's. Although the overall 
group difference on this measure is not .statistically signi- 
ficant, the consistent PDC decrement bears noting just because 
of the weight inevitably attached to reading achievement. And, 
although we performed no post-hoc significance tests of group 
differences at the site level', Figure 3e suggests that PDC 
children in California and Iowa score substantially lower in 
reading than their comparison group counterparts, even after 
adjustments have been made for WPPSI score and pther design 
factors. PDC children also score lower, overall, on subscale 
2 of the Child Rating Scale: "Aggressiveness." Figure 3g 
depifcts PDC and comparison status on this measure for all 
sites in aggregation and for the' four sites that.^ differ most 
from the main effect. For all but one of the illustrated data 
points, comparison children score higher in "Aggressiveness" 
than PDC children (in the case of the one exception, the groups 
are about equal)-. Although it is on this measure that PDC 
and comparison children differ most convincingly, it is not 
clear what meaning can be attached to th^ difference. Is ] 
aggressiveness a good or a bad quality for kindergarten chilaren. 
There are few clues to be found in Table 14, which shows that 
this scale correlates very weakly with other measures in the 
battery. And even if it is possible to make a pronouncement on 
the value of aggressiveness, is it a quality of behavior that 
PDC can be expected to influence? It is possible that group 
differences on the measure reflect something other than the 
effect of the PDC program. Considering^ the low incidence of 
significant group differences, reflected by the 68 analyses 
performed for the English-dominant sample, the indications of 
a group difference on "Aggressiveness" may simply be due to 
chance . 



^In the future, we intend to evaluate apparent^ite-level 
differences statistically and not just graphically- For the 
present, any conclusions about group differences at a given 
site are only conjectural. 



In sum, the analyses based upon data for the English- 
dominant sample provide no evidence of a general PDC-f avoring 
effect at this point. However, some of the analyses hint 
that PDC children at some sites may be making gains relative 
to comparison children while PDC children at some other sites 
may, be losing ground. Table 19 provides some insight into 
the. developing patterns. That table shows PDC-versus-comparison 
differences on the same measures graphed in Figu^^es 3a to 3g 
(but new adjusted for fall 1976 status) as being either positive 
or negative (i.e., PDC mean higher .than comparison mean or 
vice-versa) . The magnitude of these differences has not been 
evaluated statistically, and it is likely that most v/ould not 
reach statistical significance if they were evaluated. None- 
theless, since chance should favor the PDC and comparison 
groups equally, one would expect the plus and minus signs to 
canciel out if there were truly no differences between groups. 
However, at the aggregate level (all sites combined) , PDC means 
exceed compariso^ means in only 25% of the, instances shown. ^ 
Note, tht ughV that at some ^ sites the percentage is considerably 
higher wh-^le at others it is considerably lower. The sites 
seem to fall into four ranks with respect to the relative 
statuses of their PDC and .comparison group's. Proceeding downward 
in order of PDC "advantage," the ranks are: 

[Utah-{-58% PDC-favoring differences) 
West Virginia (50%) ' * 

Connecticut (42%) 
' Maryland (42%) 
Texas (42%) 

California (2^"%) 

Colorado (25%) ^ * 

Washington (25%) 

Florida (8%) 
Iowa (8%) 

Michigan (0%) ^ 



^In the' context of this table, a high "Aggressiveness" rating is 
considered positive, just as is a high PIAT-Reading score. This 
assumption is arguable, but even if the opposite assumption is 
made, it has little effect on the relative status of the sites. 
It does, however, affect the overall proportion of PDC-favoring 
differences: if a low score on that sc,ale were considered posi-. 
tive, the percentage of PDC-favoring differences would rise 
from 25 to 42. . - 



Relative 



Status of 



Table 19" 

Engl ish-Dominar>t PDC and Comparison Groups on Selected Measures 
(V\djusted for Pre-Existing Differences^) 





Verbal 
Memory-1 


Verbal 
Memory-3 


POCL-1 : 
"Task 
Orien- 
tation" 


PIAT- 
Math 


PIAT- 
Reading 


CRS-1: 
"Self- 
Assurance" 


CRS-2: 
Aggres- 
siveness" 


% of Positive 
Di fferences 

(PDC > 
Coiiipari son) 




C 7 7 
Off 


C7P 
D f 0 


Q7 7 
Off 


o f 0 


877 878 


878 


S78 


Off 


LJ f U 


S77 878 


All Sites Combined 
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- 




42 
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b b 


3ff 


Utah 






















58 


Washington 






















25 


West Virginia 


+ 








' J- ' 












50 


% Qf Positive Site- 
Level Differences 
(PDC > Comparison) 


36 


45 


18 


73 


36 18 


45. 


• 0 


78 


11 


0 0 





^Group means adjusted for F76 WPPSI, F76 status on the dependent variable (where applicable), ethnicity, sex, 
and prior pre^school experience. ^ 
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Data insufficient for analysis. 
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+ PDC Mean > Comparison Mean 
- Comparison hean > PDC Mean 
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The Spanish-dominant analytic sainple . The results of 
impact analyses for the Spanish-doininant analytic sample are 
presented in Table 20. That table ^ interpreted alongside 
Figures 4a through 4e, indicates that PDC childr-^n in this 
sample scored higher than comparison children on Arm Coordination 
and on POCL-1: "Task Orientation." A number of group-by-site 
interactions are also indicated, but these are suspect due 
to the small size of the PDC sample in California (only four 
children). Indeed, the size of the total Spanish-dominant 
sample is so small that even indications of main, effects in 
these analyses do not warrant intensive interpretation. if 
ACYF concurs with this judgment, it would then be advisable 
to reconsider the conception of the evaluation for the Spanish- 
dominant analytic sample. - Even jiow, as noted, degrees-of- 
freedom restrictions require a reduction in the complexity 
of the design framework used "in analysis of data for the 
Spanish-dominant children. Given the certainty of continued, 
attrition, it is likely that in the future further departures 
from the main analytic model will be necessary. 
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Table 20 

Summary of Analyses of Program Effects for the Spanish-Dominant Analytic Sample 



Analytic Question: 



Design Factors: 



Covariate(s) : 



Dependent \/ariable(s): 



Target Measure . 



BSM-Spanish 

Verbal Fluency 

Verbal Memory-1 

Verbal Memory-3 » 

Arm Coordination 

Draw-a-Child 

PIPS 

POCL-1 :,"Task Orientation' 
P0CL-2:"Sociability" 
CRS-1 : "Self-Assurance"3 
CRS-2: "Aggressiveness" a 
CRS-3:"Dependence"a 



Sampl e 
Sizeb 



41 
45 
45 
43 
45 
45 
45 
40 
40 
41 
46 



Analysis #1 : 

"Has a group diffevenQe 
am sen since fall of the 
Head Start year?" 



Treatment group, si te 



F76 WPPSI, F76 status 
on the target measure' 



S78 status on the 
target measure 



All Sites 
Combined 



Individual 
Sites 



No 
No 
No 
No 

No 
No 
Yes* 
No 
No 
No 
No 



No 
No 
No 
No 

Yes* 

No 
Y^s* 
Yes** 
Yes** 

No . 

No 
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Analysis ifZ: 

"Has a gi'oup diffurenae 
arisen since spx'ing of 
the Head Start year?" 



Treatment group, site 



F76 WPPSI, F76^ , 
and S7/ status on 
the target measure 



S78 status on the 
• target measure 



All Sites 
Combined 

No 
No 

No 

No 
Yes** 

No 

No 
Yes* 

No 



Individual 
Sites 



No 
No 
No 
■ No 
No 
No 
Yes* 
Yes** 
Yes** 
-Analyses not 
-Analyses not 
-Analyses not 



Analysis #3: 

"Is there a difference 
in the cjrowth curves 
of the two groups?" 



Treatment group, site 



F76 WPPSI 



F76, S77, and S78 
status on the 
target measure 



All Sites 
Combined 



Individual 
Sites 



No 
No 
No 
No 
Yes* 
No 
No 
No 
No 

appl icabl e- 
appl icable- 
appl icabl e- 



No 

No 

No 
•No 

No 

No 
Yes* 

No 

No 



^Measures available for S78 only. 



< .05 (two-tailed) 

< .01 (two- tailed) 



^Number of PDC and comparison children with complete data on 
all variables involved. 

•^If adnini stored in F7u, 



Figure 4a 



fllusxration of Main Program Effects and Program -by- Si tb 
Interactions for Arm Cooraination (Spanisti-Dominanz Anai'/^:c Sample) 
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Figure 4b \^ 



lllustrnon of Main Program Effects ana Pragram-ay-Slte Interactions for Preschool Interpersonal 
Problem Solving (PIPS) (Spanish-Dominant Analytic Sample) 
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niasmvon of Main Prvgnm Effects and Progr^m-by-Site fntencxion^ for POCL-I: 
Task Orienzaticn " (S^antsh-Dominant Sarnpie) 



c 
c 

1 1 



c 



Both ZiXzs Combined 




{ndWiduai Sites 
California 



\ i i 



Texas 




r \ 1 



J L 



-MPDC 

Con-ipanson 



F76 S77 



F76 S77 S78 



F76 S77 S78 
Testing Period 



Figure 4d 

^Mlustravon of Main Program Effecrs ana Program-by -Site interacxions for POCL'2: 
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Figure 4e 



/// - von of Main Program Effects and Rrogram-bv-Site Interactions for CRS-3: 
^Depenaence"' (Soanish-Dominant Analytic Sample j 
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V 

SUMMARY 



Characteristics of the Sample . ' 

I . 

Comparability and representativeness of groups ,. The samples 
of English-dominant f.DC and comparison children . remain remarkably 
alike in terms of their characteristics at the time of program 
entry, (fall 1976); attrition does not^appear to haVe altered 
their comparability. Neither has it affected the basic* makeup 
of either group with respect to the original PDC and compariscpn 
groups; .the fpresent samples of English-dominant- children remain 
quite representative of th^ samples as they existed at ^ the 
beginning of the Head Start year. ' The Spanish-dominant samples 
of PDC and comparison children also seem to satisfy^^he criteria 
of comparability and representativeness. But these groups 
are so stnall now that it is difficult to be confident of such 
, statistical conclusions'. _ ^ 

. Adequacy of sample size . 'Among PDC children, the rate of 
attrition is higher than projected, but' this is balanced by 
a rate amor^g comparison children that is lower p[\an projected. 
Thus, overall attrition at this point! is quite close to cpur 
estimates. ' (The actual loss rate amounts to 36%, as against 
our March 1977 prediction of 39%.4) However, since West Virginia 
will no longer be pa'rticipating in PDC, this entire ;>site will 
be lost to the evaluation an future years. Still, the .number 
of PDC children likely to remain available through the end of 
third grade — about 20 per site—is large enough to represent 
the educational effects of PDC when sites are aggregated. 
The number :of comparison children is lik€ly to remain approxi- 
mately .equal to" the niamber of PDC children, providing an 
adequate sample for statistical comparisons, at least, fpr 
analyses thl^t focus on the English-dominant children. The 
Spanish-domiinant sample has diminished to the point where 
fl;onclusions| based upon statistical analyses must acknowledge 
*a "large elejuent of doubt. 

Characy^r^tics of the Measure s 

I 

The balsic battery . The instruments in the battery adminis- 
tered to Eniglish-dominant children continue to reflect' high 
internal consistency and, altogether, a factor structjire 
congruent with the intended structure . In the battery administered 
to Spanish-dominant children, some measures show signs pf dimin- 
i.-^hing reliability. This may be a consequence of the diminishing 
size of this sample of children. ^' 



ERIC 



72 



9u 



The Classroom Observation System, Phase I version . Our 
analyses of the data produced by this version of the observation, 
system cast ^oubt on the system's suitability for measuring 
either individual behavior or classroom processes. Our specula- 
tive conclusion is^-that the brief samplings of behavior provided, 
by this vei*e'ics)n of the system were not adequate to represent 
the complex r^^ture of classroom processes. Thus, further 
interpretation of these data is unwarranted. A new classroom 
observation system has been developed for Phase II of the PDC 
evaluation, and this instrument is expected to provide deeper 
and more reliable insightf into classroom processes than^ was 
possible with its predecessor. 



Impact/ of thfj PDC Program 

jfepact at all sites considered . in aggregation • Three 
series^ of analyses were undertaken to address three variations 
of the question, "Has PDC had an impact on children?" Even 
though the three analyses addressed somewhat different issues, 
and therefore would not necessarily produce identical results 
for any given measure of impact, it is when the analyses do 
agree that we can be most confident about our conclusions . 
For the sample of English-dominant children, the main analytic 
sample, there was no measure among all of those examined that 
seemed to be consistently affected by PDC (or by the treatments 
that comparison chilu^reii. experienced , either). The measure 
for which the strongest case can be made in regard to program 
impact is the "Aggressiveness" subscale of the Child Ratin(5 
Scale. One analysis indicated that there was a general difference 
between PDC and comparison children^j)n this variable, measured 
in spring 1977 and spring 1978; the^ other two analyses indicated 
that there was a group difference on this variable in at least 
some sites, but that the difference was not constant for all 
sites. It is difficult to say whether this' result constitutes 
a. positive impact of PDC, since it is uncertain how the quality 
of aggressiveness should be valued in the context of elementary 
education. On the^ basis of the analyses, it seems safe to 
say that PDC children rate lower in "Aggressiveness" than 
comparison children in some sites, if not in all sites. But 
is this result to be interpreted as positive'', negative, or 
neither? Further, whatever the value placed on aggressiveness, 
one m^st ask whether the group difference found on this variable 
can plausibly: l)e. attributed to the impact of PDC* In the 
context of the finding that the PDC and comparison groups do 
not generally differ in a consistent way on the other variables 
examined, it seems quite possible that the difference between 
groups on "Aggressiveness" is due simply to chance rather 
than to a systematic program effect. 



Impact at individual sites > While there is no persuasive 
evidence that the PBC program in general has had an educational 
impact on children as of their kindergarten year, it is pcpssible 
that the program has begun to produce an impact in some sites > 
In Utah and West Virginia, particularly, we found differences 
favoring the PDC group on a number of measures. . And there 
are oth^r sites, in contrast, where the differences seem to 
favor the comparison children. ° We cannot state that these 
differences' are statistically significant, however; nor can 
we say that they are necessarily due to the educational program 
as distinct from other factors. Our analyses did not pursue 
these issues to a high degrefe of certainty because of the 
lack of systematic information 'on the multitude of factors 
potentially influencing the ^ impact PDC has. For this reason, 
the current report is intended only an interim 'document 
for 'circulation, within ACYF. In the future, the question of 
.PDC's impact at individuaT sites wil4. be investigated in ways 
intended to provide firmer statistical support for conclusions 
concerning site-to-site differences in program impact. And 
in documenting program impact , whether for all sites or for 
individual sites, we will also strive to determine just what 
dimensions of the program (including such factors as classrgom 
environment, teacher attitudes and parent involvement) account 
for its Impact. 
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" APPENDIX A 



Description of the Measures in the,- Spring 1978 Battery 

• \^ 

Order of 
AdTninistration 

• Classroom-Level Measures 

PDC Classroom Observation System " 1 . 



• Social-Emotional Measures 

- Preschool Interpersonal Problem- 
Solving Test (PIPS) 6 
Pupil Observation Checklist (POCL) 9 
' PDC Child Rating Scale 



Psychomotor Measures 

Arm Coordination ' [McCarthy Scales of 
Children' s Ability (MSCA) ] 



Cognitive and Language Measures 

Bilingual Syntax Measure (BSm) 2 
Block Design (WPPSI)^ ■ administered fall 1976 

Verbal Memory (MSCA) 3 

Verbal Fluency (MSCA) 5 

Draw-A-Child (MSCA) '- 4 ^ 
Peabody Individual Achievement Test 
(PIAT)-Math & Reading Recognition 

Subtests 8 



Each of these measures is described briefly below. For 
a more extensive review, see Interim Report II, Part B : 
Recommendations for Measuring Program Impact (1975 ) . 



^As noted in the text, the battery was administered in One 
or sometimes two sessions.. 

^WPPSI serves as an index of general cognitive aptitude, 
and Tvas administered as a baseline measure at ail sites in 
fall 1976. It was administered at only four sites in 
spring 1977. This was for the purpose of estimating its 
test-retest stability and sensitivity to change. 




Bilingual Syntax Measure (Burt, bulay and Hernandez Ch,, 
1975 This test xs designed to measure children's oral 
*>proficiency in English and/or Spanish grammatical . structures . 
Simple questions are used with cartoon-type colored pictures 
to provide a conversational setting for eliciting natural 
speech. ^ An analysis of the child's response yields a num.erical 
indicator and a qualitative description of the child '-s 
structural language proficiency in standard English or 
standard Spanish. Responses are written down verbatim. 

Children 'who show facility- in both Spanish and English 
receive both versions of the BSM. The order in which the 
two versions are administered is controlled so that during 
any single testing- period half the children receive the^ 
Spajiish version first and half receive the English version 
first. 

^ McCarthy Scales of Children's Abilities (McCarthy, 1972) ^ . 
These subtests consist of a series of tasks tapping problem- 
solving, psychomotor, and concepiijial abilities, and are simi- 
lar to the Wechsler Scales, but with emphasis on age-related 
maturational indicators. 

• Verbal Memory . The child is asked to repeat 
sequences of wards (Verbal Memory-1) and to 
repeat or retell as much as possible of a one- 
paragraph story (Verbal Memory-3) . McCarthy's 
Scale 1 is not used in the PDC evaluation. 

Verbal Fluency . The child is asked to name as 
many mehibers of specific categories (e.g., animals) 
as he/she can. 

• Arm Coordination . Child bounces a rubber ball, 
catches a beanbag:, and throws a beanbag through 
a hole in a target. 

• Draw-A-Child . Child draws a picture of a child 
of the same sex. 
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^Burt, M. , Dulay, H., & Hernandez Chavez, E. Bilingual 
Syntax Measure . New York: Harcourt, Brace, Jovanovich, 
1975. 

2 Mc C arthy , D . McCarthy Scales of Children's Abilities; Manual . 
New York: Psychological Corporation, 1972.' 
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PDC Child Rating Scale (High/Scope Foundation, unpublished ) . 
This instrument, designed as a measure of the construct "social 
competence," is completed ^by each child's classroom teacher. 
The scale consists of items that refer to aspects of school 
^ s behavior, such as "USes words or "Wits to influence others," and 

the teacher rates the frequency with which the child demonstrates 
this behavior C^V ;y frequently" to "Rarely"). :.As noted^-in the 
text, the 39 items- of the original Child Rating Scale have been 
reduced to 17 (comprising the scales "Self-Assurance," 
"Aggressiveness," and "Dependence") and three n^w items have 
been added to constitute an "Academic Motivation" scale (to be 
administered from spring 1979 onward) . 

PDC' Classroom Observation System (High/gcope Foundation , 
unpublished ) . The PDC observation system was developed to 
provide information about children's classroom behavior 
along dimensions pertinent to the social-emotional goals of 
Project Developmental Continuity. The system focuses on aspects 
of an individual child's behavior, verbal or nonverbal, that 
reflect the child's attitude toward himself, and on the child's 
social competence as demonstrated in his interaction with peers 
and adults. 

Using a time-sampling method, trained observers observe 
each child for five minutes at two different times during the 
day and code the observed behavior into four general categories: 
"noninvolved, " "involved," "interacts with peer," and "inter- 
act? with adult." A fifth category, "activity level," is 
included to provide information concerning the context in which 
these behaviors were observed. Each of these categories includes 
subcategories that are designed to identify the frequency and 
nature of specific behaviors -within the general category. 

Peabody Individual Achievement Test (Dunn and Markwardt , 
1970 ) The math and reading recognition subtests of this 
individually administered instrioment were used to assess these 
Wo dimensions of school achievement. The math subtest taps 
skills such as computation, mastery of ordination, shape discri- 
mination, and understanding of key terms, such as "double" and 
"youngest". The examiner reads each math question aloud and the 
child responds by pointing to one of four possible answers 
displayed on a plate in the administrators ' s notebook. The 



^Dunn, L. M., & Markwardt, F. C. , Jr. Manual for the Peabody 
Individual Achievement Test . Circle Pines, Minnesota: 
American Guidance Service, 1970. 
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reading recognition subtest consists of letter, reciognition 
tasks, which involve the matching and naming of alphabetic 
letters, and. word recognition tasks, which consist of reading 
written words aloud. ~ 

Preschool Interpersonal Problem-Solving Test (Shure and 
Spivack, 1974 ) ^~ The PIPS attempts to assess rhe child's 
ability to name alternative solutions to a life-related pro- 
blem — that of obtaining a toy from another child. Paper cut- 
outs of boys, girls and toys' are used in presenting the problem. 
Among inne^ city four-year-olds attending the Philadelphia Get 
Set day care program, those judged as better-adjusted by their 
teachers were able to conceptualize a greater number and a 
wider range of alternative .solutions ^to real-life problems than - 
were their more poorly adjusted classmates. 

Pupil Observation Checklist (High/Scope Foundation , 
Tinpublished ) . This is a rating scale consisting of twelve ^ 
7-point bipolar adjectives derived from a similar scale used 
in' the Home Start evaluation.^ The tester rates each child 
using this instrument after he or she has administered all the 
other measures in the battery to the child. The two subscales 
of the POCL, ^"Task Orientation'^ (9 items) and "Sociability" 
(3, items) were derived by factor analysis. The factor structure 
has been replicated in a number of independent studies. 

Wechsler Preschool and Primary Scale of Intelligence , 
Block^Design Subtest (Wechsler, 1967 ) ^> .The task requires 
reproducing (constructing) designs with flat colored blocks, 
either from the examiner's model or fromc^a picture on a card. 
The measure taps problem-solving abilities, flexibility of 
response style, visual-motor organization, and execution. 
The WPPSI was administered only in fall 1976, with the intention 
that this measure would continue to serve as a covariate in 
future analyses. 



^Shure, M. B. , & Spivack, G. The PIPS Test Manual > Philadel- 
phia: Hahneman Medical College, 1974. , 

^Love, J., et al. National Home Start Evaluation Interim 
Report VII : Ypsilanti, MI: High/Scope Foundation, March 
1976. 

^Wechsler, nD. Wechsler Preschool and Primary Scale of Intel - 
ligence; Manual . New York: Psychological Corporation^ 1967. 



APPENDIX B 



Significance of the Contributions of Covariates and Design 
Factors in Impact Analysis Series 1, 2, and 3, 



For the sake of documenting the roles of the covariates 
and design factors entering into the analyses of PDC's impact, 
Tables B-1, B-2 and B-3 indicate the significance levels 
attained by each term in the model for each analysis. It 
would be preferable to include in these tables estimates of 
the magnitude of the contribution (e.g., coefficients), but 
these estimates are unfortunately not recoverable from the out- 
put of the computer program used. y 



Included additionally in Table B-3 'is the significance 
level associated with a test for linear trend in each of the 
analyses performed. This is a test of the hypothesis that the 
dependent measures ' describe a line that increases or decreases 
(rather than remaining constant) over time. , 
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Table B-1 

Significance of the Contributions of Cov^rlates and Design Factors In Analysis Series #1 

(English-Dominant Analytic Sample) . 
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^Measure fiot administered at thi^s timepoint. WPPSI alone was used as a covariate in this analysis. 
NS = nonsignificant 



^probability of associated F ratio < .05 

^^probability of associated F ratio < .01 

*^*probdbility of associated F ratio < .001 
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•Table B-2 ' , . , ' ' 

Significance of the Contributions of Covarlates and Design Factors in.AnalysIs Series #2 

(Er^gl ish-Dominant Analytic Sample) 
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^Measure not administered at this timepoint^ WPPSI and S77 score on the same CRS scale were used as the 
covariates in this analysis. 
NS = nonsignificant 
♦probability of associated' F ratio < .05 
**probabil ity of associated F ratio < .01 
O *probability of associated F ratio < .001 lOo 
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" Table g-3 

Significance of the Contributions of Covarlates and Design Factors In Analysis. Series' ^3 * 

(Eng)lsh-Domlnant Analyt1c*Sample) - - 
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